-- Logs begin at Thu 2022-06-23 10:17:18 UTC, end at Thu 2022-06-23 11:20:59 UTC. -- Jun 23 10:48:12.085096 dv-cinder-843309-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 23 10:48:12.097027 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 23 10:48:12 2022] *** Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: nodename: dv-cinder-843309-2-iscsi Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: machine: x86_64 Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: clock source: unix Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: pcre jit disabled Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: detected number of CPU cores: 4 Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: current working directory: / Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: detected binary path: /usr/bin/uwsgi-core Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: your processes number limit is 47680 Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: your memory page size is 4096 bytes Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: detected max file descriptor number: 2048 Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: lock engine: pthread robust mutexes Jun 23 10:48:12.097993 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: thunder lock: enabled Jun 23 10:48:12.098709 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 23 10:48:12.098709 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 23 10:48:12.142813 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: Python main interpreter initialized at 0x557c807449c0 Jun 23 10:48:12.142813 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: python threads support enabled Jun 23 10:48:12.142813 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: your server socket listen backlog is limited to 100 connections Jun 23 10:48:12.142813 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: your mercy for graceful operations on workers is 90 seconds Jun 23 10:48:12.143143 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: mapped 403077 bytes (393 KB) for 2 cores Jun 23 10:48:12.143247 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: *** Operational MODE: preforking *** Jun 23 10:48:12.143516 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: *** uWSGI is running in multiple interpreter mode *** Jun 23 10:48:12.143516 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: spawned uWSGI master process (pid: 80192) Jun 23 10:48:12.143736 dv-cinder-843309-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 23 10:48:12.144513 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: spawned uWSGI worker 1 (pid: 80206, cores: 1) Jun 23 10:48:12.144689 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: spawned uWSGI worker 2 (pid: 80207, cores: 1) Jun 23 10:48:12.144841 dv-cinder-843309-2-iscsi devstack@keystone.service[80192]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 23 10:48:17.597398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 10:48:17.597398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: from cryptography.utils import int_from_bytes Jun 23 10:48:17.598158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 10:48:17.598158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: from cryptography.utils import int_from_bytes Jun 23 10:48:17.598270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 10:48:17.598270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: from cryptography.utils import int_from_bytes Jun 23 10:48:17.598270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 10:48:17.598270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: from cryptography.utils import int_from_bytes Jun 23 10:48:17.647680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 10:48:17.647880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 10:48:17.648077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] command line args: [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 10:48:17.648354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 10:48:17.648538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ================================================================================ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 10:48:17.648797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] admin_token = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.649050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_dir = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.649242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.649427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_source = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.649612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] debug = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.649814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.650306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.650397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.650609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.650862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.651098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.651282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_config_append = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.651463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.651680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_dir = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.651848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.652089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.652189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 10:48:17.652343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.652438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 10:48:17.652592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.652669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] command line args: [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 10:48:17.652842 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.653022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 10:48:17.653110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.653189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.653309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ================================================================================ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 10:48:17.653432 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.653645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] admin_token = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.653722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.653910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.654011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_dir = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.654148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.654311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.654386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.654499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_source = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.654608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.654686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] debug = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.654858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.654959 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.655098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.655184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.655281 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.655364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.655544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.655819 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.655911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.656057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.656132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] publish_errors = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.656238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_config_append = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.656310 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.656417 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.656523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.656600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_dir = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.656707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.656789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.656961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.657049 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.657204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.657288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.657453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] standard_threads = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.657532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.657691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.657778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.657852 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.657926 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.658032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.658109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.658277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_journal = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.658353 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.658458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_json = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.658540 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.658639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_stderr = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.658751 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.658834 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_syslog = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.658990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.659067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.659226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.659303 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.659460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.659543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.659730 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.659817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.659982 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.660058 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.660262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.660347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.660508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.660762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.660837 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.660944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] publish_errors = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.661020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.661182 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.661259 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.661442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.661520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.661622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.661752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.661871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.661995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.662110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.662249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.662348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] standard_threads = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.662491 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.662573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.662741 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.662828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.662979 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.663061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.663157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_journal = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.663239 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.663334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_json = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.663468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.663545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_stderr = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.663692 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_syslog = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.663780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.663930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:17.664014 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.664188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.664266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.664430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.664512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.664696 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.664787 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.664940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.665021 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.665192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.665275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.665428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.665503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.665666 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.665746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.665902 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.665979 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.666143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.666224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.666391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.666467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.666641 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.666716 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.666878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.666955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.667113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.667196 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.667352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.667428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.667602 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.667691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.667866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.667943 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.668110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.668184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.668344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.668419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.668584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.668661 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.668825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.668900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.669061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.669144 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.669298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.669374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.669535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.669610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.669772 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.669856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.670013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.670089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.670266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.670343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.670499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.670581 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.670739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.670813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.670974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.671048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.671210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.671356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.671451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.671528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.671712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.671800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.671949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.672038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.672190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.672265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.672430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.672516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.672667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.672754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.672904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.672985 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.673139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.673213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.673370 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.673456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.673607 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.673684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.673841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.673909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.674076 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.674149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.674309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.674373 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.674540 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.674607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.674781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.674860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.675022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.675115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.675255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.675331 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.675492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.675575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.675748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.675823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.675992 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.676070 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.676234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.676320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.676493 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.676570 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.676727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.676805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.677490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.677572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.677730 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.677815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.677972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.678048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.678211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.678286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.678449 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.678531 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.678683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.678760 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.678922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.678997 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.679161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.679238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.679397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.679478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.679628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.679734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.679893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.679970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.680147 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.680222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.680384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.680460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.680626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.680712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.680866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.680946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.681114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.681192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.681358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.681436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.681598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.681675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.681839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.681916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.682079 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.682202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.682314 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.682403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.682545 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.682621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.682786 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.682864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.683020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.683098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.683258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.683333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.683496 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.683574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.683759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.683839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.684003 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.684081 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.684244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.684332 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.684484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.684563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.684724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.684802 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.684968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.external = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.685045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.685207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.685286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.685462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.685539 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.685706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.685791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.685952 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.password = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.686028 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.686188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.token = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.686266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.686425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.686508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.686663 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.686738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.686894 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.686969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.687130 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.687215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.687365 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.687442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.687604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.687749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.687863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.687970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.688101 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.688186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.688338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.688416 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.688590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.688669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.688832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.688917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.689071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.689148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.689319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.689405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.689560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.689645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.689794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.external = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.689870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.690030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.690108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.690307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.690395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.690555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.690631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.690797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.password = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.690874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.691032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.token = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.691117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.691275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.691353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.691514 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.691592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.691790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.691880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.692029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.692107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.692279 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.692358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.692550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.693290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.693374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.693455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.693561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.693640 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.693812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.693899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.694055 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.694132 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.694306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.694383 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.694547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.694633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.694792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.694870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.695031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.695109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.695289 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.695374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.695531 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.695610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.695809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.695888 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.696064 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.696154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.696312 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.696389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.696558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.696636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.696812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.696898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.697051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.697130 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.697295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.697373 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.697554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.697640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.697797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.697874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.698035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.698111 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.698273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.698420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.698495 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.698566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.698747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.698830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.698986 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.699064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.699237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.699315 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.699484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.699560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.699760 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.699843 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.700003 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.700080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.700256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.700335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.700496 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.700573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.700732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.700812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.700975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.701054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.701210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.701290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.701449 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.701534 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.701690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.701767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.701926 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.702005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.702169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.702247 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.702402 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.702478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.702662 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.702741 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.702896 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.702980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.703133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.703210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.703368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.703447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.703615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.703708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.703874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.703951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.704116 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.704196 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.704351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.704429 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.704597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.704682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.704836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.704915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.705083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.705161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.705323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.705401 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.705555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.705634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.705797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.705881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.706031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.706108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.706269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.706347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.706506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.706579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.706743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.706819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.706993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.707072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.707227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.707304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.707469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.707551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.707738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.707821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.707977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.708054 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.708249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.708326 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.708498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.708615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.708731 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.708809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.708971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.709048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.709205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.709285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.709444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.709522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.709684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.709768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.709922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.710006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.710158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.710235 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.710395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.710479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.710637 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.710714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.710871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.710948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.711112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.711197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.711348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.711427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.711596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.711686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.711872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.711960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.712115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.712195 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.712349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.712426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.712593 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.712678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.712832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.712908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.713071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.713154 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.713311 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.713396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.713550 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.713628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.713799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.713875 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.714047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.714131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.714291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.714368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.714528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.714605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.714762 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.714848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.715002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.715079 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.715240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.715317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.715476 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.715560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.715730 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.715811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.715982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.716058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.716228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.716318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.716460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.716537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.716701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.716779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.716944 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.717030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.717174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.717251 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.717425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.717503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.717666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.717752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.717906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.717983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.718144 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.718221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.718380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.718465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.718625 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.718703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.718864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.718940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.719103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.719187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.719339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.719415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.719582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.719672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.719840 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.719928 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.720090 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.720167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.720329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.720406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.720567 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.720653 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.720822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.720900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.721070 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.721146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.721304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.721456 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.721531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.721602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.721773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.721851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.722019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.722106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.722268 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.722345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.722509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.722586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.722758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.722835 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.723008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.723090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.723247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.723403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.723489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.723643 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.723766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.723945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.724024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.724201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.724291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.724455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.724532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.724697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.724773 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.724937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.725022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.725191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.725268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.725444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.725521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.725687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.725773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.725924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.726002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.726161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.726239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.726399 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.726485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.726651 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.726728 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.726889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.726966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.727139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.727225 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.727379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.727455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.727625 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.727720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.727883 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.727970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.728125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.728280 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.728358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.728521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.728599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.728776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.728856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.729031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.729108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.729282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.729368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.729535 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.729613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.729773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.729851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.730017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.730101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.730269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.730348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.730509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.730585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.730750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.730833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.730985 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.731061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.731221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.731299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.731470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.731553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.731733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.731815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.731983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.732061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.732222 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.732310 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.732462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.732540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.732697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.732784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.732937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.733016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.733178 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.733264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.733429 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.733507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.733670 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.733747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.733909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.733995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.734150 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.734228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.734390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.734468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.734634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.734719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.734879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.734956 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.735125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.735202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.735365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.735450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.735602 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.735692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.735869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.735945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.736115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.736203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.736590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.736673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.736841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.736917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.737078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.737163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.737237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 10:48:17.737366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.737631 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.737881 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.738133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.738376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.738620 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.738860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.739100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.739339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.740125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.740377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.740624 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.740874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.741115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.741372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.741612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.741860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.742093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.742336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.742574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:17.742747 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 10:48:17.768399 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.768630 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.768857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.769594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.769831 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.770060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.777516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.778627 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.806591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.806824 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.807257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.807492 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.948612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.948786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.998217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.998455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.998977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:17.999217 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.062707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.062929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.063020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.063151 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.063252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.063372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.063454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.063588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.063728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.063834 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.063924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.064056 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.088565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.088701 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.088800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.088893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.089030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.089101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.089256 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.089326 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.089475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.089546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.089692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.089759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.089906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.089982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.103124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.103866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.116696 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.116935 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.117050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.117189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.117283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.117401 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.117491 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.117616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.117707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.117822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.117912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.118027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.118117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.118259 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:18.135165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.135380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 10:48:18.135748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.135835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.135955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 10:48:18.136039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 10:48:18.136227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.136309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.136488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.136569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 10:48:18.136749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.136830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.137004 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.137083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.137253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.137504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.148344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.148623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.148774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.148893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.149048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.149175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.149317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.149594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.157448 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.158175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.160757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.161042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.161855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.162032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.162149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.162308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.162433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.162584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.162695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.162859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.162949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.163135 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.163215 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.163391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.163471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.163686 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.163782 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.163949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.164030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.164217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.164488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.164750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.190291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.191201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.191915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.192197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.192871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.193148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.197436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.197832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.198116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.198351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.198791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.199029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.205344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.205629 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.206701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.206984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.210602 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.210893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.211178 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.211462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.211758 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.211977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.212069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.212263 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.212354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.212546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.212649 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.212824 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.213094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.213363 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.213630 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.213905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.231476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.231780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.232974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.233250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.236549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.238072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.240215 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.240610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.241711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.242105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.245615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.247084 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.249227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.250659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.252890 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.253192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.253482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.253770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.254048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.254302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.254406 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.254602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.254887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.255195 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.255470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.255774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.273763 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.274058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.274325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.274594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.275109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.275398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.275678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.275948 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.282457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.283747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.284188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.284474 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.285444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.285719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.288202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.288480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.288757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.289032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.289369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.289497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.289785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.290334 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.290643 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.290976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.302579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.302859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.303150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.303431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.304127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.304405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.304693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.304948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.305077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.305247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.305550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.306456 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.306747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.307041 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.323449 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.324782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.326759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.328056 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.330088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.331311 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.331842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.333059 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.333987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.334269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.334550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.335193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.335470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.335776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.341860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.343049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.343469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.344797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.346714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.347002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.347393 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.347697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.348044 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.348330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.348739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.349568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.357244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.357534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.357813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.358554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.358837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.359114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.365922 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.366205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.366488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.366765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.367036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.367208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:18.367425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.367555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.367746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.367878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.368029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.368155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.368283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.368418 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.368566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.368703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.368842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.369019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.369148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.369276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.369405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.369533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.369668 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.369797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.369956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.370216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.370487 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.370758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:18.407390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557c807449c0 pid: 80206 (default app) Jun 23 10:48:18.408652 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557c807449c0 pid: 80207 (default app) Jun 23 10:48:18.410770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd7915-b5b4-422a-a26d-78ac94c8b3ea None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:18.411064 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd7915-b5b4-422a-a26d-78ac94c8b3ea None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:18.411942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd7915-b5b4-422a-a26d-78ac94c8b3ea None None] PATH_INFO: `/v3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:18.416156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1/1] 10.222.0.65 () {50 vars in 791 bytes} [Thu Jun 23 10:48:18 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:26.023913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34d1d4-c633-44eb-b1f7-b4bb9f8ec1e9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:26.024508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34d1d4-c633-44eb-b1f7-b4bb9f8ec1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:26.024508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34d1d4-c633-44eb-b1f7-b4bb9f8ec1e9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:26.028871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1/2] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:26.034185 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049ba37-fd84-4f5c-91ba-bda9851868d8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:26.034622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049ba37-fd84-4f5c-91ba-bda9851868d8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:26.034754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049ba37-fd84-4f5c-91ba-bda9851868d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:26.103235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: INFO dbcounter [None req-e049ba37-fd84-4f5c-91ba-bda9851868d8 None None] Registered counter for database keystone Jun 23 10:48:26.124311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG oslo_db.sqlalchemy.engines [None req-e049ba37-fd84-4f5c-91ba-bda9851868d8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80206) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 10:48:26.131153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writer thread running {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 10:48:26.303705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG passlib.handlers.bcrypt [None req-e049ba37-fd84-4f5c-91ba-bda9851868d8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80206) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 10:48:26.304051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG passlib.handlers.bcrypt [None req-e049ba37-fd84-4f5c-91ba-bda9851868d8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80206) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 10:48:26.357929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e049ba37-fd84-4f5c-91ba-bda9851868d8 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:26.359519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e049ba37-fd84-4f5c-91ba-bda9851868d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:26.478116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 2/3] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 10:48:26.633561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34d1d4-c633-44eb-b1f7-b4bb9f8ec1e9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:26.702169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: INFO dbcounter [None req-fb34d1d4-c633-44eb-b1f7-b4bb9f8ec1e9 None None] Registered counter for database keystone Jun 23 10:48:26.720951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG oslo_db.sqlalchemy.engines [None req-fb34d1d4-c633-44eb-b1f7-b4bb9f8ec1e9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80207) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 10:48:26.727027 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writer thread running {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 10:48:26.846013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34d1d4-c633-44eb-b1f7-b4bb9f8ec1e9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:26.850243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1c18e-980a-445f-a213-ae30656e89db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:26.850899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1c18e-980a-445f-a213-ae30656e89db admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:26.851184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1c18e-980a-445f-a213-ae30656e89db admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:26.851461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1c18e-980a-445f-a213-ae30656e89db admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:26.859445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1c18e-980a-445f-a213-ae30656e89db admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:26.910488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d2e1c18e-980a-445f-a213-ae30656e89db admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 10:48:26.910775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1c18e-980a-445f-a213-ae30656e89db admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:26.920102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 2/4] 10.222.0.65 () {58 vars in 1324 bytes} [Thu Jun 23 10:48:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 10:48:26.925947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049ba37-fd84-4f5c-91ba-bda9851868d8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:26.940230 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049ba37-fd84-4f5c-91ba-bda9851868d8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:26.945189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b5daa-554d-4b06-ab82-94f6d2b7bb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:26.946296 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b5daa-554d-4b06-ab82-94f6d2b7bb9d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:26.946586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b5daa-554d-4b06-ab82-94f6d2b7bb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:26.946867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b5daa-554d-4b06-ab82-94f6d2b7bb9d admin admin] PATH_INFO: `/v3/projects/06f70799381a401b8e11df5b00b4192c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:26.950920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b5daa-554d-4b06-ab82-94f6d2b7bb9d admin admin] RBAC: Authorizing `identity:get_project(project_id=06f70799381a401b8e11df5b00b4192c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:27.000626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e42b5daa-554d-4b06-ab82-94f6d2b7bb9d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 10:48:27.000971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b5daa-554d-4b06-ab82-94f6d2b7bb9d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:27.006099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 3/5] 10.222.0.65 () {56 vars in 1207 bytes} [Thu Jun 23 10:48:26 2022] GET /identity/v3/projects/06f70799381a401b8e11df5b00b4192c => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:28.973781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aaf2f4-5699-40b2-892b-ecc7fae60fa6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:28.974319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aaf2f4-5699-40b2-892b-ecc7fae60fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:28.974319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aaf2f4-5699-40b2-892b-ecc7fae60fa6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:28.978442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 3/6] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:28.983712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b5e53-9191-4551-9fa0-50e25763a504 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:28.984087 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b5e53-9191-4551-9fa0-50e25763a504 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:28.984283 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b5e53-9191-4551-9fa0-50e25763a504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:29.003378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b96b5e53-9191-4551-9fa0-50e25763a504 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:29.005552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b96b5e53-9191-4551-9fa0-50e25763a504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:29.013400 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 4/7] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 10:48:29.036244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aaf2f4-5699-40b2-892b-ecc7fae60fa6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:29.048048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aaf2f4-5699-40b2-892b-ecc7fae60fa6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:29.052816 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ea7d2-ebde-4e68-a1ec-7f8c1753b7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:29.053436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ea7d2-ebde-4e68-a1ec-7f8c1753b7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:29.053718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ea7d2-ebde-4e68-a1ec-7f8c1753b7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:29.053996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ea7d2-ebde-4e68-a1ec-7f8c1753b7e9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:29.061566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ea7d2-ebde-4e68-a1ec-7f8c1753b7e9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:29.063323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ea7d2-ebde-4e68-a1ec-7f8c1753b7e9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:29.072058 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 4/8] 10.222.0.65 () {58 vars in 1324 bytes} [Thu Jun 23 10:48:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 10:48:29.077197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b5e53-9191-4551-9fa0-50e25763a504 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:29.086074 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b5e53-9191-4551-9fa0-50e25763a504 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:29.090439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce6e7d-2971-4326-a290-8af2ca804531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:29.091333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6e7d-2971-4326-a290-8af2ca804531 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:29.091621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6e7d-2971-4326-a290-8af2ca804531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:29.091932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6e7d-2971-4326-a290-8af2ca804531 admin admin] PATH_INFO: `/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:29.093324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce6e7d-2971-4326-a290-8af2ca804531 admin admin] RBAC: Authorizing `identity:get_user(user_id=7c47db9f01c84c63b57cf3ae5711ffb1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:29.095139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce6e7d-2971-4326-a290-8af2ca804531 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:29.096738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 5/9] 10.222.0.65 () {56 vars in 1198 bytes} [Thu Jun 23 10:48:29 2022] GET /identity/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:31.137626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175aafb-096a-4b94-8136-76e62320f65c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:31.138153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175aafb-096a-4b94-8136-76e62320f65c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:31.138241 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175aafb-096a-4b94-8136-76e62320f65c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:31.139529 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 5/10] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:31.144519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e20de-7d8e-4248-9ba6-ce7d7c3287b1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:31.144947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e20de-7d8e-4248-9ba6-ce7d7c3287b1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:31.145174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e20de-7d8e-4248-9ba6-ce7d7c3287b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:31.161424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6c8e20de-7d8e-4248-9ba6-ce7d7c3287b1 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:31.162588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6c8e20de-7d8e-4248-9ba6-ce7d7c3287b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:31.206507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877886e-3f1a-4302-bbdd-b26eff6ede6b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:31.206822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877886e-3f1a-4302-bbdd-b26eff6ede6b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:31.207104 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877886e-3f1a-4302-bbdd-b26eff6ede6b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:31.208486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 6/11] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:48:31.213660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38aadc-2ac0-44ab-8dbf-c43c85ba70aa None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:31.214013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38aadc-2ac0-44ab-8dbf-c43c85ba70aa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:31.214378 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38aadc-2ac0-44ab-8dbf-c43c85ba70aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:31.242784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 6/12] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:31.255060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG passlib.handlers.bcrypt [None req-0a38aadc-2ac0-44ab-8dbf-c43c85ba70aa None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80207) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 10:48:31.255490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG passlib.handlers.bcrypt [None req-0a38aadc-2ac0-44ab-8dbf-c43c85ba70aa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80207) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 10:48:31.265963 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e20de-7d8e-4248-9ba6-ce7d7c3287b1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:31.289089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:31.301738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0a38aadc-2ac0-44ab-8dbf-c43c85ba70aa None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:31.303330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0a38aadc-2ac0-44ab-8dbf-c43c85ba70aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:31.368171 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e20de-7d8e-4248-9ba6-ce7d7c3287b1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:31.401562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 7/13] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:31.402040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a103d2e-3fe8-4ab0-ad02-181905d123c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:31.402838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a103d2e-3fe8-4ab0-ad02-181905d123c5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:31.403123 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a103d2e-3fe8-4ab0-ad02-181905d123c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:31.403399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a103d2e-3fe8-4ab0-ad02-181905d123c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:31.414810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a103d2e-3fe8-4ab0-ad02-181905d123c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:31.424166 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38aadc-2ac0-44ab-8dbf-c43c85ba70aa None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1a103d2e-3fe8-4ab0-ad02-181905d123c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:31.426920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:31.427838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:48:31.428658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 7/14] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:48:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:48:31.433957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a103d2e-3fe8-4ab0-ad02-181905d123c5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:31.514571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:31.523323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38aadc-2ac0-44ab-8dbf-c43c85ba70aa None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:31.525762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:31.533305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a103d2e-3fe8-4ab0-ad02-181905d123c5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:31.562248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64d084-d20d-4ca2-b1f9-2e0accdb58a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:31.563146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64d084-d20d-4ca2-b1f9-2e0accdb58a3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:31.563440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64d084-d20d-4ca2-b1f9-2e0accdb58a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:31.563752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64d084-d20d-4ca2-b1f9-2e0accdb58a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:31.567142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce06346-21d5-4582-9a1b-d0d1bb4413e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:31.567846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce06346-21d5-4582-9a1b-d0d1bb4413e2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:31.568133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce06346-21d5-4582-9a1b-d0d1bb4413e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:31.568412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce06346-21d5-4582-9a1b-d0d1bb4413e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:31.569164 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce06346-21d5-4582-9a1b-d0d1bb4413e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:31.569475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce06346-21d5-4582-9a1b-d0d1bb4413e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:31.570866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce06346-21d5-4582-9a1b-d0d1bb4413e2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:31.582293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 8/15] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:31.587632 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce06346-21d5-4582-9a1b-d0d1bb4413e2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:31.631420 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64d084-d20d-4ca2-b1f9-2e0accdb58a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:31.632901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64d084-d20d-4ca2-b1f9-2e0accdb58a3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:31.685900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce06346-21d5-4582-9a1b-d0d1bb4413e2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:31.718395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2429c9-84e9-4e9a-86be-f11b27d8424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:31.719264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2429c9-84e9-4e9a-86be-f11b27d8424f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:31.719547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2429c9-84e9-4e9a-86be-f11b27d8424f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:31.719846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2429c9-84e9-4e9a-86be-f11b27d8424f None admin] PATH_INFO: `/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:31.721035 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2429c9-84e9-4e9a-86be-f11b27d8424f None admin] RBAC: Authorizing `identity:get_user(user_id=7c47db9f01c84c63b57cf3ae5711ffb1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:31.723284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2429c9-84e9-4e9a-86be-f11b27d8424f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:31.725080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 9/16] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:31 2022] GET /identity/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:31.730081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2429c9-84e9-4e9a-86be-f11b27d8424f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:31.730477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 8/17] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:48:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:48:31.735952 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64d084-d20d-4ca2-b1f9-2e0accdb58a3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:31.738678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:31.744665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:31.829168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2429c9-84e9-4e9a-86be-f11b27d8424f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:31.833336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64d084-d20d-4ca2-b1f9-2e0accdb58a3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:31.862624 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7ff11-f5a5-4cd2-b9b5-debfb48b45c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:31.863439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7ff11-f5a5-4cd2-b9b5-debfb48b45c2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:31.863745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7ff11-f5a5-4cd2-b9b5-debfb48b45c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:31.864027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7ff11-f5a5-4cd2-b9b5-debfb48b45c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:31.865195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7ff11-f5a5-4cd2-b9b5-debfb48b45c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:31.866639 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7ff11-f5a5-4cd2-b9b5-debfb48b45c2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:31.867717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fda485-653b-4e68-aa82-0425e51dc8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:31.868264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 10/18] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:31.868526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fda485-653b-4e68-aa82-0425e51dc8bb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:31.868792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fda485-653b-4e68-aa82-0425e51dc8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:31.869073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fda485-653b-4e68-aa82-0425e51dc8bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:31.873845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7ff11-f5a5-4cd2-b9b5-debfb48b45c2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:31.887251 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fda485-653b-4e68-aa82-0425e51dc8bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:31.888678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fda485-653b-4e68-aa82-0425e51dc8bb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-93fda485-653b-4e68-aa82-0425e51dc8bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:48:31.909160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:31.910546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:31.911338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:48:31.911338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:48:31.911338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:48:31.911338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:48:31.911338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 9/19] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:48:31.916656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fda485-653b-4e68-aa82-0425e51dc8bb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:31.955167 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:31.971554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:31.975849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7ff11-f5a5-4cd2-b9b5-debfb48b45c2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:32.009464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ad534-f623-4f2d-9a92-0ae5995504ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:32.010115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ad534-f623-4f2d-9a92-0ae5995504ad None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:32.010400 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ad534-f623-4f2d-9a92-0ae5995504ad None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:32.010676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ad534-f623-4f2d-9a92-0ae5995504ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:32.011533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ad534-f623-4f2d-9a92-0ae5995504ad None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.domain.id=default, user.id=7c47db9f01c84c63b57cf3ae5711ffb1) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:32.011894 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ad534-f623-4f2d-9a92-0ae5995504ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:32.013338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ad534-f623-4f2d-9a92-0ae5995504ad None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:32.014886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fda485-653b-4e68-aa82-0425e51dc8bb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:32.020794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 11/20] 10.222.0.65 () {56 vars in 1322 bytes} [Thu Jun 23 10:48:31 2022] GET /identity/v3/role_assignments?user.id=7c47db9f01c84c63b57cf3ae5711ffb1&scope.domain.id=default&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:32.050468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66034ba4-b246-459d-8e22-ffb3404f9578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:32.051159 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66034ba4-b246-459d-8e22-ffb3404f9578 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:32.051439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66034ba4-b246-459d-8e22-ffb3404f9578 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:32.051746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66034ba4-b246-459d-8e22-ffb3404f9578 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:32.052497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66034ba4-b246-459d-8e22-ffb3404f9578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:32.052808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66034ba4-b246-459d-8e22-ffb3404f9578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:32.054244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66034ba4-b246-459d-8e22-ffb3404f9578 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:32.074100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 10/21] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:34.239175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b8690-e76f-4c09-82dc-5dd0d33b0f1d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:34.240044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b8690-e76f-4c09-82dc-5dd0d33b0f1d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:34.240488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b8690-e76f-4c09-82dc-5dd0d33b0f1d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:34.242028 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 12/22] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:34.247506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b7b51-0367-4c0f-adbf-e6217ae9f703 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:34.248262 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b7b51-0367-4c0f-adbf-e6217ae9f703 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:34.248682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b7b51-0367-4c0f-adbf-e6217ae9f703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:34.272685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-912b7b51-0367-4c0f-adbf-e6217ae9f703 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:34.274127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-912b7b51-0367-4c0f-adbf-e6217ae9f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:34.359481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 11/23] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:34.394716 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b8690-e76f-4c09-82dc-5dd0d33b0f1d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:34.533796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b8690-e76f-4c09-82dc-5dd0d33b0f1d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:34.577586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b468d87-1437-4a78-85a2-670ef29c5855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:34.578395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b468d87-1437-4a78-85a2-670ef29c5855 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:34.578683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b468d87-1437-4a78-85a2-670ef29c5855 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:34.578965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b468d87-1437-4a78-85a2-670ef29c5855 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:34.592649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b468d87-1437-4a78-85a2-670ef29c5855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5b468d87-1437-4a78-85a2-670ef29c5855 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:34.600338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:34.601168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:34.602355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:34.602355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:34.602355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:34.602355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:34.602355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:34.602355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:34.602355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:34.602355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:34.602355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:34.602355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:34.602355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:34.602355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:48:34.603211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12289302-d36c-40b8-ad19-439a01ec3c40 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:34.603211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12289302-d36c-40b8-ad19-439a01ec3c40 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:34.603211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12289302-d36c-40b8-ad19-439a01ec3c40 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:34.603383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 13/24] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:48:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:34.603447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 12/25] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:34.607554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b468d87-1437-4a78-85a2-670ef29c5855 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:34.612226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8013f-9e84-43d0-afb3-0da80a6a5cda None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:34.612455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8013f-9e84-43d0-afb3-0da80a6a5cda None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:34.613272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8013f-9e84-43d0-afb3-0da80a6a5cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:34.620000 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:34.664736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f4d8013f-9e84-43d0-afb3-0da80a6a5cda None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:34.665957 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f4d8013f-9e84-43d0-afb3-0da80a6a5cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:34.768469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:34.772195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b468d87-1437-4a78-85a2-670ef29c5855 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:34.813335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 13/26] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:34.842313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def2918-ecb2-4238-85d1-211400192364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:34.843432 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def2918-ecb2-4238-85d1-211400192364 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:34.843929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def2918-ecb2-4238-85d1-211400192364 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:34.844400 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def2918-ecb2-4238-85d1-211400192364 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:34.845371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def2918-ecb2-4238-85d1-211400192364 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:34.845867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def2918-ecb2-4238-85d1-211400192364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:34.847483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def2918-ecb2-4238-85d1-211400192364 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:34.850340 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8013f-9e84-43d0-afb3-0da80a6a5cda None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:34.864043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 14/27] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:34.869127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def2918-ecb2-4238-85d1-211400192364 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:35.007787 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8013f-9e84-43d0-afb3-0da80a6a5cda None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:35.035166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:35.056252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305280f1-8c45-4b8b-9a97-59c5a777bf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:35.057563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305280f1-8c45-4b8b-9a97-59c5a777bf13 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.058391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305280f1-8c45-4b8b-9a97-59c5a777bf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.059133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305280f1-8c45-4b8b-9a97-59c5a777bf13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.074094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305280f1-8c45-4b8b-9a97-59c5a777bf13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:35.076429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305280f1-8c45-4b8b-9a97-59c5a777bf13 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:35.080630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def2918-ecb2-4238-85d1-211400192364 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-305280f1-8c45-4b8b-9a97-59c5a777bf13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:48:35.102284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:35.103108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:35.103938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:48:35.103938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:48:35.103938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:48:35.103938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:48:35.105998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 14/28] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:35.107546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbde7d8-2273-43b3-9e2e-1a819f4bc62e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.108040 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbde7d8-2273-43b3-9e2e-1a819f4bc62e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.108480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbde7d8-2273-43b3-9e2e-1a819f4bc62e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.109922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 15/29] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:35.111414 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea48f3-089c-4674-afaf-cf76890bea48 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.112022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea48f3-089c-4674-afaf-cf76890bea48 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.112556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea48f3-089c-4674-afaf-cf76890bea48 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.114551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 16/30] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:35.115562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea48f3-089c-4674-afaf-cf76890bea48 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:35.134887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc76114-1a7e-47fa-95ce-76cc247a6c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:35.135801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc76114-1a7e-47fa-95ce-76cc247a6c7e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.136084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc76114-1a7e-47fa-95ce-76cc247a6c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.136358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc76114-1a7e-47fa-95ce-76cc247a6c7e None admin] PATH_INFO: `/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.137672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc76114-1a7e-47fa-95ce-76cc247a6c7e None admin] RBAC: Authorizing `identity:get_user(user_id=7c47db9f01c84c63b57cf3ae5711ffb1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:35.139250 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc76114-1a7e-47fa-95ce-76cc247a6c7e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:35.141340 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 15/31] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:34 2022] GET /identity/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1 => generated 247 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:35.142818 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed06a6-215a-44b1-9e8c-1bfbe33948a9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.143281 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed06a6-215a-44b1-9e8c-1bfbe33948a9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.143779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed06a6-215a-44b1-9e8c-1bfbe33948a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.148669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:35.164614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e2ed06a6-215a-44b1-9e8c-1bfbe33948a9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:35.166479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e2ed06a6-215a-44b1-9e8c-1bfbe33948a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:35.229110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea48f3-089c-4674-afaf-cf76890bea48 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:35.242732 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 16/32] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:35.244744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b49bf-546f-4f54-9cea-d558c6c49fe9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.245244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b49bf-546f-4f54-9cea-d558c6c49fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.245797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b49bf-546f-4f54-9cea-d558c6c49fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.263128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-560b49bf-546f-4f54-9cea-d558c6c49fe9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:35.264688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-560b49bf-546f-4f54-9cea-d558c6c49fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:35.266772 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b33935-3073-4223-9ce7-61a7c70b8031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:35.267548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b33935-3073-4223-9ce7-61a7c70b8031 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.268008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b33935-3073-4223-9ce7-61a7c70b8031 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.268367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b33935-3073-4223-9ce7-61a7c70b8031 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.269908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b33935-3073-4223-9ce7-61a7c70b8031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:35.269908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b33935-3073-4223-9ce7-61a7c70b8031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:35.271043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b33935-3073-4223-9ce7-61a7c70b8031 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:35.283587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 17/33] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:35.287897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b33935-3073-4223-9ce7-61a7c70b8031 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:35.337242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:35.348796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 17/34] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:35.349944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b49bf-546f-4f54-9cea-d558c6c49fe9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:35.381158 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:35.394562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b33935-3073-4223-9ce7-61a7c70b8031 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:35.431214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d81042-74b7-401e-905a-39bd97cfca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:35.432111 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d81042-74b7-401e-905a-39bd97cfca1e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.432469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d81042-74b7-401e-905a-39bd97cfca1e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.432817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d81042-74b7-401e-905a-39bd97cfca1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.434613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d81042-74b7-401e-905a-39bd97cfca1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:35.435990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d81042-74b7-401e-905a-39bd97cfca1e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:35.437819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 18/35] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:35.438739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d81042-74b7-401e-905a-39bd97cfca1e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:35.462165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b49bf-546f-4f54-9cea-d558c6c49fe9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:35.499980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049c83a-46d3-4186-b122-97249215289c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:35.500941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049c83a-46d3-4186-b122-97249215289c None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.501455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049c83a-46d3-4186-b122-97249215289c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.501921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049c83a-46d3-4186-b122-97249215289c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.503035 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049c83a-46d3-4186-b122-97249215289c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:35.504750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049c83a-46d3-4186-b122-97249215289c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:35.548848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d81042-74b7-401e-905a-39bd97cfca1e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:35.584284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b967087-d829-4ae2-aac3-3d955e81860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:35.584955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b967087-d829-4ae2-aac3-3d955e81860a None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.585245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b967087-d829-4ae2-aac3-3d955e81860a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.585527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b967087-d829-4ae2-aac3-3d955e81860a None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.586248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b967087-d829-4ae2-aac3-3d955e81860a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:35.587727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b967087-d829-4ae2-aac3-3d955e81860a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:35.654722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 18/36] 10.222.0.65 () {60 vars in 1131 bytes} [Thu Jun 23 10:48:35 2022] POST /identity/v3/roles => generated 219 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:35.656302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049c83a-46d3-4186-b122-97249215289c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:35.720507 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:35.754108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 19/37] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:48:35 2022] POST /identity/v3/projects => generated 301 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:35.755175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b967087-d829-4ae2-aac3-3d955e81860a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:35.771811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:35.788562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049c83a-46d3-4186-b122-97249215289c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:35.830423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66f38e-89bb-4bc9-b582-7c7646cc5fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:35.832271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66f38e-89bb-4bc9-b582-7c7646cc5fdf None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.832765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66f38e-89bb-4bc9-b582-7c7646cc5fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.833257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66f38e-89bb-4bc9-b582-7c7646cc5fdf None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.834402 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66f38e-89bb-4bc9-b582-7c7646cc5fdf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:35.836269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66f38e-89bb-4bc9-b582-7c7646cc5fdf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:35.855242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 19/38] 10.222.0.65 () {60 vars in 1131 bytes} [Thu Jun 23 10:48:35 2022] POST /identity/v3/roles => generated 225 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:35.857590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b1d7c-0f35-48ff-9c73-9dc34e3418ce None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.858153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b1d7c-0f35-48ff-9c73-9dc34e3418ce None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.858637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b1d7c-0f35-48ff-9c73-9dc34e3418ce None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.862521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 20/39] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:35.868211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2443845-08a2-4018-8d35-a4995b73392c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:35.869845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2443845-08a2-4018-8d35-a4995b73392c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:35.870580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2443845-08a2-4018-8d35-a4995b73392c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:35.894842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e2443845-08a2-4018-8d35-a4995b73392c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:35.897800 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e2443845-08a2-4018-8d35-a4995b73392c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:35.955014 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b967087-d829-4ae2-aac3-3d955e81860a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:35.997142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 21/40] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:36.028016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5beb6b6-76c8-48f1-814e-3a86af6faf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:36.029244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beb6b6-76c8-48f1-814e-3a86af6faf7f None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:36.029754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beb6b6-76c8-48f1-814e-3a86af6faf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:36.030392 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beb6b6-76c8-48f1-814e-3a86af6faf7f None admin] PATH_INFO: `/v3/domains/default/users/7c47db9f01c84c63b57cf3ae5711ffb1/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:36.033290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5beb6b6-76c8-48f1-814e-3a86af6faf7f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7c47db9f01c84c63b57cf3ae5711ffb1, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:36.035169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5beb6b6-76c8-48f1-814e-3a86af6faf7f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:36.041344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2443845-08a2-4018-8d35-a4995b73392c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:36.063269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 20/41] 10.222.0.65 () {58 vars in 1361 bytes} [Thu Jun 23 10:48:35 2022] PUT /identity/v3/domains/default/users/7c47db9f01c84c63b57cf3ae5711ffb1/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:48:36.146549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69300c6f-16b8-4025-b67a-acb7a748cf5c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:36.146846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69300c6f-16b8-4025-b67a-acb7a748cf5c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:36.147137 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69300c6f-16b8-4025-b67a-acb7a748cf5c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:36.148832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 21/42] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:48:36.156166 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0a313-f231-4f66-b986-5c8f5a664652 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:36.156470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0a313-f231-4f66-b986-5c8f5a664652 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:36.156740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0a313-f231-4f66-b986-5c8f5a664652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:36.188412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-86d0a313-f231-4f66-b986-5c8f5a664652 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:36.190055 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-86d0a313-f231-4f66-b986-5c8f5a664652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:36.236147 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:36.270667 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:36.277605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 22/43] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:36.300584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2443845-08a2-4018-8d35-a4995b73392c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:36.301076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0a313-f231-4f66-b986-5c8f5a664652 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:36.337643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7070c-210a-45ca-92e4-0f90ff3d1646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:36.338327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7070c-210a-45ca-92e4-0f90ff3d1646 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:36.338616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7070c-210a-45ca-92e4-0f90ff3d1646 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:36.338906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7070c-210a-45ca-92e4-0f90ff3d1646 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:36.339680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7070c-210a-45ca-92e4-0f90ff3d1646 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:36.341085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7070c-210a-45ca-92e4-0f90ff3d1646 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:36.354992 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 22/44] 10.222.0.65 () {60 vars in 1131 bytes} [Thu Jun 23 10:48:36 2022] POST /identity/v3/roles => generated 223 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:36.412903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0a313-f231-4f66-b986-5c8f5a664652 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:36.450535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7bd88-a5dc-41c8-9a73-bbdc041b7c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:36.451419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7bd88-a5dc-41c8-9a73-bbdc041b7c96 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:36.451744 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7bd88-a5dc-41c8-9a73-bbdc041b7c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:36.452033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7bd88-a5dc-41c8-9a73-bbdc041b7c96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:36.453289 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7bd88-a5dc-41c8-9a73-bbdc041b7c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:36.454776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7bd88-a5dc-41c8-9a73-bbdc041b7c96 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:36.457274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 23/45] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:36.462737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7070c-210a-45ca-92e4-0f90ff3d1646 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:36.570739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7070c-210a-45ca-92e4-0f90ff3d1646 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:36.573067 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:36.605487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1ac41-fd8f-4ed0-80b5-b23df948df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:36.606154 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1ac41-fd8f-4ed0-80b5-b23df948df1c None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:36.606442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1ac41-fd8f-4ed0-80b5-b23df948df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:36.606725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1ac41-fd8f-4ed0-80b5-b23df948df1c None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:36.607450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1ac41-fd8f-4ed0-80b5-b23df948df1c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:36.608992 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1ac41-fd8f-4ed0-80b5-b23df948df1c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:36.635718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 23/46] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:48:36 2022] POST /identity/v3/projects => generated 312 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:38.107702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec15c55-5ca4-4654-9c31-daa9d20431c3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:38.108105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec15c55-5ca4-4654-9c31-daa9d20431c3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:38.108265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec15c55-5ca4-4654-9c31-daa9d20431c3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:38.109788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 24/47] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:38.115151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfe5ed-75fe-417d-9b84-3ef70e354719 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:38.115526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfe5ed-75fe-417d-9b84-3ef70e354719 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:38.115755 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfe5ed-75fe-417d-9b84-3ef70e354719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:38.132857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0abfe5ed-75fe-417d-9b84-3ef70e354719 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:38.134052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0abfe5ed-75fe-417d-9b84-3ef70e354719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:38.198997 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 24/48] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:38.222698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec15c55-5ca4-4654-9c31-daa9d20431c3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:38.287397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:38.321867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec15c55-5ca4-4654-9c31-daa9d20431c3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:38.354995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952adb1-65e4-4f0d-a5e5-64ea228c8c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:38.355746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952adb1-65e4-4f0d-a5e5-64ea228c8c50 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:38.356032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952adb1-65e4-4f0d-a5e5-64ea228c8c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:38.356310 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952adb1-65e4-4f0d-a5e5-64ea228c8c50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:38.367909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9952adb1-65e4-4f0d-a5e5-64ea228c8c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9952adb1-65e4-4f0d-a5e5-64ea228c8c50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:38.376499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:38.377387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:48:38.378762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 25/49] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:48:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:38.384151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfe5ed-75fe-417d-9b84-3ef70e354719 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:38.466386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:38.481703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfe5ed-75fe-417d-9b84-3ef70e354719 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:38.515571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4e8a3-d4f5-4d22-85b6-24bcc8cadf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:38.516323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4e8a3-d4f5-4d22-85b6-24bcc8cadf01 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:38.516614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4e8a3-d4f5-4d22-85b6-24bcc8cadf01 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:38.516899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4e8a3-d4f5-4d22-85b6-24bcc8cadf01 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:38.517664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4e8a3-d4f5-4d22-85b6-24bcc8cadf01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:38.517980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4e8a3-d4f5-4d22-85b6-24bcc8cadf01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:38.519414 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4e8a3-d4f5-4d22-85b6-24bcc8cadf01 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:38.527773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 25/50] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:38.533204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952adb1-65e4-4f0d-a5e5-64ea228c8c50 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:38.633545 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952adb1-65e4-4f0d-a5e5-64ea228c8c50 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:38.653462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a2545-d204-4522-acd6-e7f69b5bfa0f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:38.653759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a2545-d204-4522-acd6-e7f69b5bfa0f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:38.654047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a2545-d204-4522-acd6-e7f69b5bfa0f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:38.655471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 26/51] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:48:38.661238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6fa26-0e9c-4f10-80e7-6d9c2fd283c1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:38.661532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6fa26-0e9c-4f10-80e7-6d9c2fd283c1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:38.661823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6fa26-0e9c-4f10-80e7-6d9c2fd283c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:38.679941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bbb6fa26-0e9c-4f10-80e7-6d9c2fd283c1 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:38.681183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bbb6fa26-0e9c-4f10-80e7-6d9c2fd283c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:38.754807 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 27/52] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:38.764710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:38.773710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48872e37-45dd-43b9-a800-b2b699635916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:38.774588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48872e37-45dd-43b9-a800-b2b699635916 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:38.774876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48872e37-45dd-43b9-a800-b2b699635916 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:38.775156 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48872e37-45dd-43b9-a800-b2b699635916 None admin] PATH_INFO: `/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:38.776362 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48872e37-45dd-43b9-a800-b2b699635916 None admin] RBAC: Authorizing `identity:get_user(user_id=7c47db9f01c84c63b57cf3ae5711ffb1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:38.777823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48872e37-45dd-43b9-a800-b2b699635916 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:38.777989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6fa26-0e9c-4f10-80e7-6d9c2fd283c1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:38.779458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 26/53] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:38 2022] GET /identity/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:38.785185 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48872e37-45dd-43b9-a800-b2b699635916 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:38.857659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:38.880393 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6fa26-0e9c-4f10-80e7-6d9c2fd283c1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:38.886508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48872e37-45dd-43b9-a800-b2b699635916 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:38.913734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15f411-7d7c-495b-b918-9caf2a9fd11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:38.914539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15f411-7d7c-495b-b918-9caf2a9fd11a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:38.914826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15f411-7d7c-495b-b918-9caf2a9fd11a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:38.915102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15f411-7d7c-495b-b918-9caf2a9fd11a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:38.916306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15f411-7d7c-495b-b918-9caf2a9fd11a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:38.917759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15f411-7d7c-495b-b918-9caf2a9fd11a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:38.919378 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3974129-64fb-4c32-baae-5a39d6d922d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:38.919511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 28/54] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:38.920210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3974129-64fb-4c32-baae-5a39d6d922d1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:38.920504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3974129-64fb-4c32-baae-5a39d6d922d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:38.920788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3974129-64fb-4c32-baae-5a39d6d922d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:38.921986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3974129-64fb-4c32-baae-5a39d6d922d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:38.923421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3974129-64fb-4c32-baae-5a39d6d922d1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:38.925411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15f411-7d7c-495b-b918-9caf2a9fd11a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:38.930551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 27/55] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:38.931347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3974129-64fb-4c32-baae-5a39d6d922d1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:38.992766 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:39.022749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15f411-7d7c-495b-b918-9caf2a9fd11a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:39.031676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3974129-64fb-4c32-baae-5a39d6d922d1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:39.056619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a1038-9f5d-4052-8ecb-bcf716e19d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:39.057282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1038-9f5d-4052-8ecb-bcf716e19d3c None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:39.057572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1038-9f5d-4052-8ecb-bcf716e19d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:39.057859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1038-9f5d-4052-8ecb-bcf716e19d3c None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:39.058576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a1038-9f5d-4052-8ecb-bcf716e19d3c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:39.060063 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a1038-9f5d-4052-8ecb-bcf716e19d3c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:39.066101 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401a79d-936d-4fec-aa30-a7e334e2df06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:39.066750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401a79d-936d-4fec-aa30-a7e334e2df06 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:39.067037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401a79d-936d-4fec-aa30-a7e334e2df06 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:39.067317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401a79d-936d-4fec-aa30-a7e334e2df06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:39.068190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401a79d-936d-4fec-aa30-a7e334e2df06 None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.domain.id=default, user.id=7c47db9f01c84c63b57cf3ae5711ffb1) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:39.068503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401a79d-936d-4fec-aa30-a7e334e2df06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:39.069933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401a79d-936d-4fec-aa30-a7e334e2df06 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:39.078162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:39.078978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 28/56] 10.222.0.65 () {56 vars in 1322 bytes} [Thu Jun 23 10:48:38 2022] GET /identity/v3/role_assignments?user.id=7c47db9f01c84c63b57cf3ae5711ffb1&scope.domain.id=default&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:39.085565 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 29/57] 10.222.0.65 () {60 vars in 1140 bytes} [Thu Jun 23 10:48:38 2022] POST /identity/v3/projects => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:48:41.105081 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b6ff7-af11-48f5-8bd8-c56817c41751 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:41.105463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b6ff7-af11-48f5-8bd8-c56817c41751 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:41.105642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b6ff7-af11-48f5-8bd8-c56817c41751 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:41.107016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 29/58] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:41.112947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf793b-eb70-4c1c-b62d-1c6e97a3c8ba None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:41.113353 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf793b-eb70-4c1c-b62d-1c6e97a3c8ba None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:41.113524 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf793b-eb70-4c1c-b62d-1c6e97a3c8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:41.130026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c0cf793b-eb70-4c1c-b62d-1c6e97a3c8ba None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:41.131219 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c0cf793b-eb70-4c1c-b62d-1c6e97a3c8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:41.195674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 30/59] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:41.219094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b6ff7-af11-48f5-8bd8-c56817c41751 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:41.318273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b6ff7-af11-48f5-8bd8-c56817c41751 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:41.342397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:41.351724 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a94b5a-7b1e-4684-a182-4231c2085783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:41.352462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a94b5a-7b1e-4684-a182-4231c2085783 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:41.352751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a94b5a-7b1e-4684-a182-4231c2085783 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:41.353033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a94b5a-7b1e-4684-a182-4231c2085783 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:41.354208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a94b5a-7b1e-4684-a182-4231c2085783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:41.355679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a94b5a-7b1e-4684-a182-4231c2085783 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:41.357325 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 30/60] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:41.363292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf793b-eb70-4c1c-b62d-1c6e97a3c8ba None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:41.458002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf793b-eb70-4c1c-b62d-1c6e97a3c8ba None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:41.483501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:41.491518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b07100-5bfb-443a-9a02-9576a9971577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:41.492216 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b07100-5bfb-443a-9a02-9576a9971577 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:41.492507 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b07100-5bfb-443a-9a02-9576a9971577 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:41.492792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b07100-5bfb-443a-9a02-9576a9971577 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:41.493519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b07100-5bfb-443a-9a02-9576a9971577 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:41.494966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b07100-5bfb-443a-9a02-9576a9971577 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:41.519530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 31/61] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:48:41 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:44.729467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab504b1a-2885-4462-a756-2505ebbc957c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:44.730358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab504b1a-2885-4462-a756-2505ebbc957c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:44.730790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab504b1a-2885-4462-a756-2505ebbc957c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:44.732384 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 31/62] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:44.738326 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c639b8-d2e8-4e36-a4d9-dd32cea2f1e9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:44.738787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c639b8-d2e8-4e36-a4d9-dd32cea2f1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:44.739078 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c639b8-d2e8-4e36-a4d9-dd32cea2f1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:44.760458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-96c639b8-d2e8-4e36-a4d9-dd32cea2f1e9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:44.761706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-96c639b8-d2e8-4e36-a4d9-dd32cea2f1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:44.791228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5c54a-13a0-47b6-8ef0-4dad44978cff None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:44.791890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5c54a-13a0-47b6-8ef0-4dad44978cff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:44.792379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5c54a-13a0-47b6-8ef0-4dad44978cff None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:44.793990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 32/63] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:44.799013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc70210-6f7d-487d-a677-7973f20281d1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:44.799598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc70210-6f7d-487d-a677-7973f20281d1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:44.800104 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc70210-6f7d-487d-a677-7973f20281d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:44.818981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1bc70210-6f7d-487d-a677-7973f20281d1 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:44.820581 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1bc70210-6f7d-487d-a677-7973f20281d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:44.857208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 32/64] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:44.858703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7566ad-d8b8-4f98-90e8-3c497e1668d6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:44.859000 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7566ad-d8b8-4f98-90e8-3c497e1668d6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:44.859284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7566ad-d8b8-4f98-90e8-3c497e1668d6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:44.860844 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 33/65] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:44.866702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3632d-c8c3-4f4d-91d5-4eb49032d544 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:44.867206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3632d-c8c3-4f4d-91d5-4eb49032d544 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:44.867703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3632d-c8c3-4f4d-91d5-4eb49032d544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:44.894129 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-44e3632d-c8c3-4f4d-91d5-4eb49032d544 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:44.895344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-44e3632d-c8c3-4f4d-91d5-4eb49032d544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:44.910569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 33/66] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:44.916725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc70210-6f7d-487d-a677-7973f20281d1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:44.967339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 34/67] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:44.968531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3632d-c8c3-4f4d-91d5-4eb49032d544 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.016394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.062896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc70210-6f7d-487d-a677-7973f20281d1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.075701 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3632d-c8c3-4f4d-91d5-4eb49032d544 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.090753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.104349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb40bc-e933-4f56-a28a-1cf258b9808a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.105163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb40bc-e933-4f56-a28a-1cf258b9808a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.105455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb40bc-e933-4f56-a28a-1cf258b9808a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.105744 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb40bc-e933-4f56-a28a-1cf258b9808a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.114439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdb7ce-5225-4e39-a9ff-feb62287f37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.115457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdb7ce-5225-4e39-a9ff-feb62287f37e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.115783 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdb7ce-5225-4e39-a9ff-feb62287f37e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.116099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdb7ce-5225-4e39-a9ff-feb62287f37e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.117165 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb40bc-e933-4f56-a28a-1cf258b9808a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-70cb40bc-e933-4f56-a28a-1cf258b9808a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:45.124868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:45.125781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:48:45.126547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 34/68] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:48:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:45.128581 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb40bc-e933-4f56-a28a-1cf258b9808a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.132001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdb7ce-5225-4e39-a9ff-feb62287f37e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-95bdb7ce-5225-4e39-a9ff-feb62287f37e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:45.139823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:45.140646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:48:45.141412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 35/69] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:45.142922 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bc016-1b8c-49d6-9f1d-396a47e289d2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.142922 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bc016-1b8c-49d6-9f1d-396a47e289d2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.142922 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bc016-1b8c-49d6-9f1d-396a47e289d2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.144444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 36/70] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:45.145304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bc016-1b8c-49d6-9f1d-396a47e289d2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.236437 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb40bc-e933-4f56-a28a-1cf258b9808a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.260774 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bc016-1b8c-49d6-9f1d-396a47e289d2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.264183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.271517 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd80af-373d-47dd-a6c8-2c771301014a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.272297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd80af-373d-47dd-a6c8-2c771301014a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.272580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd80af-373d-47dd-a6c8-2c771301014a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.272859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd80af-373d-47dd-a6c8-2c771301014a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.283409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd80af-373d-47dd-a6c8-2c771301014a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-dacd80af-373d-47dd-a6c8-2c771301014a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:45.291103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:45.291983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:48:45.292720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 35/71] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:45.293351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd80af-373d-47dd-a6c8-2c771301014a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.304470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b6489-89f6-428e-bf93-c08a0050239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.305154 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b6489-89f6-428e-bf93-c08a0050239e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.305445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b6489-89f6-428e-bf93-c08a0050239e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.305733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b6489-89f6-428e-bf93-c08a0050239e None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.306500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b6489-89f6-428e-bf93-c08a0050239e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:45.306818 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b6489-89f6-428e-bf93-c08a0050239e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.308337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b6489-89f6-428e-bf93-c08a0050239e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.317130 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 37/72] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:45.318319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1344e1-24a8-4aae-99e9-53b746159711 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.318618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1344e1-24a8-4aae-99e9-53b746159711 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.318906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1344e1-24a8-4aae-99e9-53b746159711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.334581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3e1344e1-24a8-4aae-99e9-53b746159711 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:45.335817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3e1344e1-24a8-4aae-99e9-53b746159711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:45.339530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.393590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd80af-373d-47dd-a6c8-2c771301014a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.401882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 38/73] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:45.402722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1344e1-24a8-4aae-99e9-53b746159711 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.427768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d7588-fe35-40e3-98b0-a5608e80d336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.428450 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d7588-fe35-40e3-98b0-a5608e80d336 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.428741 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d7588-fe35-40e3-98b0-a5608e80d336 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.429025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d7588-fe35-40e3-98b0-a5608e80d336 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.429787 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d7588-fe35-40e3-98b0-a5608e80d336 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:45.430101 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d7588-fe35-40e3-98b0-a5608e80d336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.431498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d7588-fe35-40e3-98b0-a5608e80d336 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.443866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 36/74] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:45.445110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac81eab-3316-4cfd-ba7a-78213390cd4c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.445404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac81eab-3316-4cfd-ba7a-78213390cd4c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.445683 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac81eab-3316-4cfd-ba7a-78213390cd4c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.447090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 37/75] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:45.448890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac81eab-3316-4cfd-ba7a-78213390cd4c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.495164 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.501998 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1344e1-24a8-4aae-99e9-53b746159711 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.536220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0254a-43ba-42c0-b1a2-94795a338a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.536830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0254a-43ba-42c0-b1a2-94795a338a67 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.537118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0254a-43ba-42c0-b1a2-94795a338a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.537402 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0254a-43ba-42c0-b1a2-94795a338a67 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.538161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c0254a-43ba-42c0-b1a2-94795a338a67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:45.538476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c0254a-43ba-42c0-b1a2-94795a338a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.539908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c0254a-43ba-42c0-b1a2-94795a338a67 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.547919 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 39/76] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:45.547919 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0254a-43ba-42c0-b1a2-94795a338a67 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.552984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac81eab-3316-4cfd-ba7a-78213390cd4c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.558076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.586748 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66585d-2062-48d2-ae48-e164ffd136d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.587610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66585d-2062-48d2-ae48-e164ffd136d1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.587927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66585d-2062-48d2-ae48-e164ffd136d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.588202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66585d-2062-48d2-ae48-e164ffd136d1 None admin] PATH_INFO: `/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.589398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66585d-2062-48d2-ae48-e164ffd136d1 None admin] RBAC: Authorizing `identity:get_user(user_id=7c47db9f01c84c63b57cf3ae5711ffb1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.590855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66585d-2062-48d2-ae48-e164ffd136d1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.592501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 38/77] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:45.593261 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66585d-2062-48d2-ae48-e164ffd136d1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.650484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0254a-43ba-42c0-b1a2-94795a338a67 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.683030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cbc156-c9ab-47b8-b8d8-3cc9b03df2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.683862 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbc156-c9ab-47b8-b8d8-3cc9b03df2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.684157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbc156-c9ab-47b8-b8d8-3cc9b03df2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.684433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbc156-c9ab-47b8-b8d8-3cc9b03df2e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.689336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66585d-2062-48d2-ae48-e164ffd136d1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.694750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cbc156-c9ab-47b8-b8d8-3cc9b03df2e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f6cbc156-c9ab-47b8-b8d8-3cc9b03df2e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:45.702721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:45.703566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:48:45.704435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 40/78] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:45.706058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1ffee-36be-4e08-a0ec-e2afa4a57e3b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.706153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1ffee-36be-4e08-a0ec-e2afa4a57e3b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.706402 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1ffee-36be-4e08-a0ec-e2afa4a57e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.723192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6fd1ffee-36be-4e08-a0ec-e2afa4a57e3b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:45.724402 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6fd1ffee-36be-4e08-a0ec-e2afa4a57e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:45.727825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661b953-7f85-4df9-87f5-e17149add214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.728704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661b953-7f85-4df9-87f5-e17149add214 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.728992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661b953-7f85-4df9-87f5-e17149add214 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.729269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661b953-7f85-4df9-87f5-e17149add214 None admin] PATH_INFO: `/v3/users/95e96acf6a87449d9aa6f29de04b192f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.737420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.745445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d661b953-7f85-4df9-87f5-e17149add214 None admin] RBAC: Authorizing `identity:get_user(user_id=95e96acf6a87449d9aa6f29de04b192f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.746896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d661b953-7f85-4df9-87f5-e17149add214 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.748619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 39/79] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/users/95e96acf6a87449d9aa6f29de04b192f => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:45.749394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661b953-7f85-4df9-87f5-e17149add214 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.781515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.793474 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 41/80] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:45.794782 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1ffee-36be-4e08-a0ec-e2afa4a57e3b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.853193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661b953-7f85-4df9-87f5-e17149add214 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.887065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b801053-a8c2-40c3-8378-a7c209783192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.887982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b801053-a8c2-40c3-8378-a7c209783192 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.888269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b801053-a8c2-40c3-8378-a7c209783192 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.888546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b801053-a8c2-40c3-8378-a7c209783192 None admin] PATH_INFO: `/v3/users/95e96acf6a87449d9aa6f29de04b192f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.889746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b801053-a8c2-40c3-8378-a7c209783192 None admin] RBAC: Authorizing `identity:get_user(user_id=95e96acf6a87449d9aa6f29de04b192f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.891208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b801053-a8c2-40c3-8378-a7c209783192 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.892617 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1ffee-36be-4e08-a0ec-e2afa4a57e3b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.893055 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 40/81] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/users/95e96acf6a87449d9aa6f29de04b192f => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:45.893648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b801053-a8c2-40c3-8378-a7c209783192 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.929756 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7d18b-7720-4a93-a187-e2e6b91a1ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.930412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7d18b-7720-4a93-a187-e2e6b91a1ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.930699 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7d18b-7720-4a93-a187-e2e6b91a1ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.930979 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7d18b-7720-4a93-a187-e2e6b91a1ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.949069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.993306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.996592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b801053-a8c2-40c3-8378-a7c209783192 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.998050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7d18b-7720-4a93-a187-e2e6b91a1ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.999537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7d18b-7720-4a93-a187-e2e6b91a1ec7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:46.030367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543dac3d-8b2a-4844-9d6f-bd9011c6c1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:46.031033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dac3d-8b2a-4844-9d6f-bd9011c6c1ff None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:46.031316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dac3d-8b2a-4844-9d6f-bd9011c6c1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:46.031598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dac3d-8b2a-4844-9d6f-bd9011c6c1ff None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:46.032368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543dac3d-8b2a-4844-9d6f-bd9011c6c1ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:46.032675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543dac3d-8b2a-4844-9d6f-bd9011c6c1ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:46.034061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543dac3d-8b2a-4844-9d6f-bd9011c6c1ff None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:46.041733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 41/82] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:46.042504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543dac3d-8b2a-4844-9d6f-bd9011c6c1ff None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:46.101243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 42/83] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:48:46.102100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7d18b-7720-4a93-a187-e2e6b91a1ec7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:46.144014 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543dac3d-8b2a-4844-9d6f-bd9011c6c1ff None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:46.150012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:46.177767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54da2c-8052-46b7-99b7-7893fa234523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:46.178421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54da2c-8052-46b7-99b7-7893fa234523 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:46.178711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54da2c-8052-46b7-99b7-7893fa234523 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:46.179021 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54da2c-8052-46b7-99b7-7893fa234523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:46.200433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7d18b-7720-4a93-a187-e2e6b91a1ec7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:46.208739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:46.234883 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac05d9-6344-4b4a-a532-d90a412e1e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:46.235731 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac05d9-6344-4b4a-a532-d90a412e1e0d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:46.236036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac05d9-6344-4b4a-a532-d90a412e1e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:46.236295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac05d9-6344-4b4a-a532-d90a412e1e0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:46.237509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac05d9-6344-4b4a-a532-d90a412e1e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:46.238953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac05d9-6344-4b4a-a532-d90a412e1e0d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:46.240609 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 43/84] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:46.241454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac05d9-6344-4b4a-a532-d90a412e1e0d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:46.247339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54da2c-8052-46b7-99b7-7893fa234523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:46.248830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54da2c-8052-46b7-99b7-7893fa234523 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:46.339031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac05d9-6344-4b4a-a532-d90a412e1e0d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:46.345944 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 42/85] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:48:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:48:46.346710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54da2c-8052-46b7-99b7-7893fa234523 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:46.356118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:46.373946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faee5c-87a7-4d1b-b414-245918f17805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:46.374801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faee5c-87a7-4d1b-b414-245918f17805 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:46.375276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faee5c-87a7-4d1b-b414-245918f17805 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:46.375557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faee5c-87a7-4d1b-b414-245918f17805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:46.406903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:46.442347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1faee5c-87a7-4d1b-b414-245918f17805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:46.443866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1faee5c-87a7-4d1b-b414-245918f17805 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:46.445318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54da2c-8052-46b7-99b7-7893fa234523 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:46.477980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086d632-5bdf-4acb-a20a-579e4f45bdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:46.478843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086d632-5bdf-4acb-a20a-579e4f45bdbe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:46.479132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086d632-5bdf-4acb-a20a-579e4f45bdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:46.479416 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086d632-5bdf-4acb-a20a-579e4f45bdbe None admin] PATH_INFO: `/v3/users/95e96acf6a87449d9aa6f29de04b192f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:46.480639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086d632-5bdf-4acb-a20a-579e4f45bdbe None admin] RBAC: Authorizing `identity:get_user(user_id=95e96acf6a87449d9aa6f29de04b192f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:46.482088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086d632-5bdf-4acb-a20a-579e4f45bdbe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:46.483744 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 43/86] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:46 2022] GET /identity/v3/users/95e96acf6a87449d9aa6f29de04b192f => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:46.484764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086d632-5bdf-4acb-a20a-579e4f45bdbe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:46.544580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 44/87] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:48:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:48:46.545403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faee5c-87a7-4d1b-b414-245918f17805 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:46.565657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:46.595516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086d632-5bdf-4acb-a20a-579e4f45bdbe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:46.628444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:46.636315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8036a-a468-41ba-a760-b1de459ff6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:46.637447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8036a-a468-41ba-a760-b1de459ff6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:46.637809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8036a-a468-41ba-a760-b1de459ff6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:46.638162 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8036a-a468-41ba-a760-b1de459ff6c6 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:46.640460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8036a-a468-41ba-a760-b1de459ff6c6 None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:46.642196 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8036a-a468-41ba-a760-b1de459ff6c6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:46.644530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 44/88] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:46 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:46.645141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8036a-a468-41ba-a760-b1de459ff6c6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:46.661686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faee5c-87a7-4d1b-b414-245918f17805 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:46.698466 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75188b06-735a-4054-92d9-55374fe3f7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:46.699260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75188b06-735a-4054-92d9-55374fe3f7aa None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:46.699613 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75188b06-735a-4054-92d9-55374fe3f7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:46.699991 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75188b06-735a-4054-92d9-55374fe3f7aa None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:46.700792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75188b06-735a-4054-92d9-55374fe3f7aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:46.702321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75188b06-735a-4054-92d9-55374fe3f7aa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:46.722735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 45/89] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:48:46 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:46.723664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75188b06-735a-4054-92d9-55374fe3f7aa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:46.763613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8036a-a468-41ba-a760-b1de459ff6c6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:46.809150 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f21a82-3a92-421a-922a-4af6d07daaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:46.810388 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f21a82-3a92-421a-922a-4af6d07daaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:46.810859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f21a82-3a92-421a-922a-4af6d07daaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:46.811317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f21a82-3a92-421a-922a-4af6d07daaa6 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:46.812893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f21a82-3a92-421a-922a-4af6d07daaa6 None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:46.814676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f21a82-3a92-421a-922a-4af6d07daaa6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:46.815682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:46.817855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 45/90] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:46 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:46.818852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f21a82-3a92-421a-922a-4af6d07daaa6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:46.859061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75188b06-735a-4054-92d9-55374fe3f7aa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:46.903017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:46.912594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174692b-76ff-430e-95e2-7f758863c3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:46.913536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174692b-76ff-430e-95e2-7f758863c3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:46.913690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174692b-76ff-430e-95e2-7f758863c3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:46.914093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174692b-76ff-430e-95e2-7f758863c3c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:46.956659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f21a82-3a92-421a-922a-4af6d07daaa6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:47.012320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb720d-4b19-484d-b794-0363887a6cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:47.013204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb720d-4b19-484d-b794-0363887a6cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:47.013538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb720d-4b19-484d-b794-0363887a6cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:47.013822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb720d-4b19-484d-b794-0363887a6cf2 None admin] PATH_INFO: `/v3/projects/cde95f4c105541eda1eb063175f7df78` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:47.015049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb720d-4b19-484d-b794-0363887a6cf2 None admin] RBAC: Authorizing `identity:get_project(project_id=cde95f4c105541eda1eb063175f7df78)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:47.016608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb720d-4b19-484d-b794-0363887a6cf2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:47.017716 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174692b-76ff-430e-95e2-7f758863c3c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:47.018389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 46/91] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:46 2022] GET /identity/v3/projects/cde95f4c105541eda1eb063175f7df78 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:47.019182 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb720d-4b19-484d-b794-0363887a6cf2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:47.019322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174692b-76ff-430e-95e2-7f758863c3c3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:47.111349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:47.146663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 46/92] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:48:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:48:47.147679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174692b-76ff-430e-95e2-7f758863c3c3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:47.150142 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb720d-4b19-484d-b794-0363887a6cf2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:47.186993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd862c-7df4-4345-825a-a6b1e8cc7fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:47.187575 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd862c-7df4-4345-825a-a6b1e8cc7fce None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:47.187898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd862c-7df4-4345-825a-a6b1e8cc7fce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:47.188182 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd862c-7df4-4345-825a-a6b1e8cc7fce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:47.189025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd862c-7df4-4345-825a-a6b1e8cc7fce None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=2f8281ddbf1345088c2c4475cad30949, user.id=7c47db9f01c84c63b57cf3ae5711ffb1) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:47.189342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd862c-7df4-4345-825a-a6b1e8cc7fce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:47.190877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd862c-7df4-4345-825a-a6b1e8cc7fce None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:47.193010 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:47.198218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 47/93] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:48:47 2022] GET /identity/v3/role_assignments?user.id=7c47db9f01c84c63b57cf3ae5711ffb1&scope.project.id=2f8281ddbf1345088c2c4475cad30949&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:47.199525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd862c-7df4-4345-825a-a6b1e8cc7fce None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:47.273232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174692b-76ff-430e-95e2-7f758863c3c3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:47.315808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f3766-e832-4fbf-803c-fe2586300cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:47.317249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f3766-e832-4fbf-803c-fe2586300cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:47.317739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f3766-e832-4fbf-803c-fe2586300cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:47.318255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f3766-e832-4fbf-803c-fe2586300cf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:47.319357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f3766-e832-4fbf-803c-fe2586300cf5 None admin] RBAC: Adding query filter params (role.id=7d15e336f8034a1fbdef48cfb7c4f717, scope.project.id=2f8281ddbf1345088c2c4475cad30949, user.id=95e96acf6a87449d9aa6f29de04b192f) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:47.319924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f3766-e832-4fbf-803c-fe2586300cf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:47.322014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f3766-e832-4fbf-803c-fe2586300cf5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:47.331367 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 47/94] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:48:47 2022] GET /identity/v3/role_assignments?user.id=95e96acf6a87449d9aa6f29de04b192f&scope.project.id=2f8281ddbf1345088c2c4475cad30949&role.id=7d15e336f8034a1fbdef48cfb7c4f717 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:47.331367 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f3766-e832-4fbf-803c-fe2586300cf5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:47.380283 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:47.443918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd862c-7df4-4345-825a-a6b1e8cc7fce None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:47.490567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d482c9-f74a-4178-8bb2-9a8d096bcb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:47.491625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d482c9-f74a-4178-8bb2-9a8d096bcb37 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:47.492203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d482c9-f74a-4178-8bb2-9a8d096bcb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:47.492675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d482c9-f74a-4178-8bb2-9a8d096bcb37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:47.494443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d482c9-f74a-4178-8bb2-9a8d096bcb37 None admin] RBAC: Adding query filter params (role.id=7d15e336f8034a1fbdef48cfb7c4f717, scope.project.id=cde95f4c105541eda1eb063175f7df78, user.id=95e96acf6a87449d9aa6f29de04b192f) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:47.494977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d482c9-f74a-4178-8bb2-9a8d096bcb37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:47.496788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d482c9-f74a-4178-8bb2-9a8d096bcb37 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:47.504009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 48/95] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:48:47 2022] GET /identity/v3/role_assignments?user.id=95e96acf6a87449d9aa6f29de04b192f&scope.project.id=cde95f4c105541eda1eb063175f7df78&role.id=7d15e336f8034a1fbdef48cfb7c4f717 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:47.555392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f3766-e832-4fbf-803c-fe2586300cf5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:47.609060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ee356-f1de-470e-b2f1-b0ed20dff37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:47.610232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ee356-f1de-470e-b2f1-b0ed20dff37d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:47.610521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ee356-f1de-470e-b2f1-b0ed20dff37d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:47.610803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ee356-f1de-470e-b2f1-b0ed20dff37d None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:47.612150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ee356-f1de-470e-b2f1-b0ed20dff37d None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:47.613651 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ee356-f1de-470e-b2f1-b0ed20dff37d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:47.615726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 48/96] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:47 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:47.621874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d482c9-f74a-4178-8bb2-9a8d096bcb37 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:47.717579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:47.776212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d482c9-f74a-4178-8bb2-9a8d096bcb37 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:47.827208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e9b7c-f442-483b-8c66-faff8778d5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:47.828237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e9b7c-f442-483b-8c66-faff8778d5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:47.828739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e9b7c-f442-483b-8c66-faff8778d5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:47.829212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e9b7c-f442-483b-8c66-faff8778d5e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:47.830799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e9b7c-f442-483b-8c66-faff8778d5e5 None admin] RBAC: Adding query filter params (role.id=8a0cfaebc67a497a98459da3f1a82ef9, scope.project.id=2f8281ddbf1345088c2c4475cad30949, user.id=95e96acf6a87449d9aa6f29de04b192f) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:47.831327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e9b7c-f442-483b-8c66-faff8778d5e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:47.832979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e9b7c-f442-483b-8c66-faff8778d5e5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:47.839304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 49/97] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:48:47 2022] GET /identity/v3/role_assignments?user.id=95e96acf6a87449d9aa6f29de04b192f&scope.project.id=2f8281ddbf1345088c2c4475cad30949&role.id=8a0cfaebc67a497a98459da3f1a82ef9 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:49.291051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead00a-8afe-4247-be16-075c22e49f9a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.292094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead00a-8afe-4247-be16-075c22e49f9a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.292633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead00a-8afe-4247-be16-075c22e49f9a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.294238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 49/98] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:49.311914 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e2b57-2e15-4a4e-8c0d-09abe4741466 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.312798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e2b57-2e15-4a4e-8c0d-09abe4741466 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.313278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e2b57-2e15-4a4e-8c0d-09abe4741466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.340618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3f4e2b57-2e15-4a4e-8c0d-09abe4741466 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:49.342123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3f4e2b57-2e15-4a4e-8c0d-09abe4741466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:49.429073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 50/99] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:49.467013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fead00a-8afe-4247-be16-075c22e49f9a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:49.601500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125e624-e46f-4601-b9ac-b171043832e4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.602184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125e624-e46f-4601-b9ac-b171043832e4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.602674 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125e624-e46f-4601-b9ac-b171043832e4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.604576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 51/100] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:49.608353 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fead00a-8afe-4247-be16-075c22e49f9a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:49.610367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f596ec-e8df-4600-afc8-635246ff7621 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.610869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f596ec-e8df-4600-afc8-635246ff7621 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.611640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:49.612090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f596ec-e8df-4600-afc8-635246ff7621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.635812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c1f596ec-e8df-4600-afc8-635246ff7621 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:49.637514 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c1f596ec-e8df-4600-afc8-635246ff7621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:49.686490 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db0dc3-fbbf-4a80-9a8d-bea373c7a069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:49.688345 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db0dc3-fbbf-4a80-9a8d-bea373c7a069 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.688998 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db0dc3-fbbf-4a80-9a8d-bea373c7a069 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.690901 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db0dc3-fbbf-4a80-9a8d-bea373c7a069 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.697923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:49.713198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db0dc3-fbbf-4a80-9a8d-bea373c7a069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-29db0dc3-fbbf-4a80-9a8d-bea373c7a069 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:49.722885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:49.723780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:49.724528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:49.724528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:49.724528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:49.724528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:49.724528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:49.724528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:49.724528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:49.724528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:49.724528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:49.724528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:49.724528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:48:49.724528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:48:49.725639 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 50/101] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:49.730776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db0dc3-fbbf-4a80-9a8d-bea373c7a069 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:49.737738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 52/102] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:49.766948 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f596ec-e8df-4600-afc8-635246ff7621 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:49.933864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db0dc3-fbbf-4a80-9a8d-bea373c7a069 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:49.939849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f596ec-e8df-4600-afc8-635246ff7621 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:49.989635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495db2a-27e0-49a0-a5fa-781a6a5aceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:49.990436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495db2a-27e0-49a0-a5fa-781a6a5aceb4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.990723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495db2a-27e0-49a0-a5fa-781a6a5aceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.991000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495db2a-27e0-49a0-a5fa-781a6a5aceb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.992880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495db2a-27e0-49a0-a5fa-781a6a5aceb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:49.994343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495db2a-27e0-49a0-a5fa-781a6a5aceb4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:50.002268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 53/103] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:50.007566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495db2a-27e0-49a0-a5fa-781a6a5aceb4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:50.019796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26388efc-b806-45c6-b189-0fcd253d8008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:50.020478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26388efc-b806-45c6-b189-0fcd253d8008 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.020765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26388efc-b806-45c6-b189-0fcd253d8008 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.021045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26388efc-b806-45c6-b189-0fcd253d8008 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.021802 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26388efc-b806-45c6-b189-0fcd253d8008 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:50.022121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26388efc-b806-45c6-b189-0fcd253d8008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:50.025707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26388efc-b806-45c6-b189-0fcd253d8008 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:50.037206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 51/104] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:48:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:50.042225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26388efc-b806-45c6-b189-0fcd253d8008 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:50.075368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:50.083309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:50.127905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495db2a-27e0-49a0-a5fa-781a6a5aceb4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:50.177986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e3c0f-8707-4f66-a72d-52ea69ba9bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:50.178960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e3c0f-8707-4f66-a72d-52ea69ba9bcf None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.179423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e3c0f-8707-4f66-a72d-52ea69ba9bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.179860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e3c0f-8707-4f66-a72d-52ea69ba9bcf None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.180730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e3c0f-8707-4f66-a72d-52ea69ba9bcf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:50.182330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e3c0f-8707-4f66-a72d-52ea69ba9bcf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:50.238263 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 54/105] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:48:50 2022] POST /identity/v3/projects => generated 302 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:50.240502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26388efc-b806-45c6-b189-0fcd253d8008 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:50.244366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885b113-7c7e-4c01-bb4e-2d1735182243 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.244856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885b113-7c7e-4c01-bb4e-2d1735182243 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.245301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885b113-7c7e-4c01-bb4e-2d1735182243 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.246859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 55/106] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:48:50.254374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c57a8-1988-4ce0-afcd-9574b7b98818 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.254880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c57a8-1988-4ce0-afcd-9574b7b98818 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.255304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c57a8-1988-4ce0-afcd-9574b7b98818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.295332 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-141c57a8-1988-4ce0-afcd-9574b7b98818 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:50.297762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-141c57a8-1988-4ce0-afcd-9574b7b98818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:50.329053 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345c71f-5026-4563-9cfb-dc7aaff08231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:50.332032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345c71f-5026-4563-9cfb-dc7aaff08231 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.332701 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345c71f-5026-4563-9cfb-dc7aaff08231 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.334018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345c71f-5026-4563-9cfb-dc7aaff08231 None admin] PATH_INFO: `/v3/users/90372fe182f149819ca0ca377fc079b2` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.359529 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345c71f-5026-4563-9cfb-dc7aaff08231 None admin] RBAC: Authorizing `identity:get_user(user_id=90372fe182f149819ca0ca377fc079b2)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:50.361808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345c71f-5026-4563-9cfb-dc7aaff08231 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:50.367771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 52/107] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:50 2022] GET /identity/v3/users/90372fe182f149819ca0ca377fc079b2 => generated 289 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:50.369922 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345c71f-5026-4563-9cfb-dc7aaff08231 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:50.444258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 56/108] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:50.447992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72f78f-4475-4b6c-ba0d-cf584b820080 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.447992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72f78f-4475-4b6c-ba0d-cf584b820080 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.447992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72f78f-4475-4b6c-ba0d-cf584b820080 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.447992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 57/109] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:50.452613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ec323-96e0-4909-bc36-a0bb160437a1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.452905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ec323-96e0-4909-bc36-a0bb160437a1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.453190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ec323-96e0-4909-bc36-a0bb160437a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.479092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cb8ec323-96e0-4909-bc36-a0bb160437a1 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:50.480870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cb8ec323-96e0-4909-bc36-a0bb160437a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:50.497620 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:50.518161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345c71f-5026-4563-9cfb-dc7aaff08231 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:50.524255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:50.562116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb75a12-21e5-4b2e-bf23-0955cdec5ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:50.562116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb75a12-21e5-4b2e-bf23-0955cdec5ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.562369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb75a12-21e5-4b2e-bf23-0955cdec5ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.562658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb75a12-21e5-4b2e-bf23-0955cdec5ce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.575370 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 58/110] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:50.576555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ec323-96e0-4909-bc36-a0bb160437a1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:50.661606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb75a12-21e5-4b2e-bf23-0955cdec5ce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:50.663301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb75a12-21e5-4b2e-bf23-0955cdec5ce3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:50.701749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ec323-96e0-4909-bc36-a0bb160437a1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:50.744025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166668a-934c-4e07-bfe5-f47dd34ea767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:50.745421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166668a-934c-4e07-bfe5-f47dd34ea767 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.745902 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166668a-934c-4e07-bfe5-f47dd34ea767 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.746391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166668a-934c-4e07-bfe5-f47dd34ea767 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.760401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166668a-934c-4e07-bfe5-f47dd34ea767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d166668a-934c-4e07-bfe5-f47dd34ea767 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:50.770956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:50.771848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:50.772604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:50.772604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:50.772604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:50.772604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:50.772604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:50.772604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:50.772604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:50.772604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:50.772604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:50.772604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:50.772604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:50.772604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:48:50.775416 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 59/111] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:48:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:50.780927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166668a-934c-4e07-bfe5-f47dd34ea767 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:50.788330 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:50.816751 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 53/112] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:48:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:48:50.818531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15f26-011e-4a12-8662-0fc9401cbacf None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.819015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15f26-011e-4a12-8662-0fc9401cbacf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.819484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15f26-011e-4a12-8662-0fc9401cbacf None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.820012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:50.821950 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 54/113] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:50.822957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15f26-011e-4a12-8662-0fc9401cbacf None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:50.895558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166668a-934c-4e07-bfe5-f47dd34ea767 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:50.931951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb07371-3da5-47f5-a95f-0f11cbea2641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:50.932718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb07371-3da5-47f5-a95f-0f11cbea2641 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.933009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb07371-3da5-47f5-a95f-0f11cbea2641 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.933363 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb07371-3da5-47f5-a95f-0f11cbea2641 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.939313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15f26-011e-4a12-8662-0fc9401cbacf None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:50.943893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb07371-3da5-47f5-a95f-0f11cbea2641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fcb07371-3da5-47f5-a95f-0f11cbea2641 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:50.954679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:50.955571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:48:50.956359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 60/114] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:50.957681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb07371-3da5-47f5-a95f-0f11cbea2641 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:50.978335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e723cc2-734f-4216-8bd9-0df5d6d52176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:50.979351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e723cc2-734f-4216-8bd9-0df5d6d52176 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.979861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e723cc2-734f-4216-8bd9-0df5d6d52176 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.980354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e723cc2-734f-4216-8bd9-0df5d6d52176 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.981292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e723cc2-734f-4216-8bd9-0df5d6d52176 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:50.981790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e723cc2-734f-4216-8bd9-0df5d6d52176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:50.983362 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e723cc2-734f-4216-8bd9-0df5d6d52176 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:50.997499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 55/115] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:50.999125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816db8b9-d298-4ccb-a70f-304434bdf74c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.999611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816db8b9-d298-4ccb-a70f-304434bdf74c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.000105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816db8b9-d298-4ccb-a70f-304434bdf74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.017052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-816db8b9-d298-4ccb-a70f-304434bdf74c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:51.018574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-816db8b9-d298-4ccb-a70f-304434bdf74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:51.064224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:51.071553 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb07371-3da5-47f5-a95f-0f11cbea2641 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:51.084262 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:51.091776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 56/116] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:51.096123 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816db8b9-d298-4ccb-a70f-304434bdf74c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:51.135463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220fa4d-1af7-4985-92d3-15b7c3e4fec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:51.136568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220fa4d-1af7-4985-92d3-15b7c3e4fec6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.136939 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220fa4d-1af7-4985-92d3-15b7c3e4fec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.137336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220fa4d-1af7-4985-92d3-15b7c3e4fec6 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.138655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220fa4d-1af7-4985-92d3-15b7c3e4fec6 None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:51.140386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220fa4d-1af7-4985-92d3-15b7c3e4fec6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:51.147545 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 61/117] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:50 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:51.147545 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220fa4d-1af7-4985-92d3-15b7c3e4fec6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:51.207405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816db8b9-d298-4ccb-a70f-304434bdf74c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:51.244048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec8561-84d9-4877-9dcf-dd331eb042f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:51.245079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec8561-84d9-4877-9dcf-dd331eb042f9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.245655 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec8561-84d9-4877-9dcf-dd331eb042f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.246077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec8561-84d9-4877-9dcf-dd331eb042f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.247499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec8561-84d9-4877-9dcf-dd331eb042f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:51.247971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec8561-84d9-4877-9dcf-dd331eb042f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:51.249969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec8561-84d9-4877-9dcf-dd331eb042f9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:51.255042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220fa4d-1af7-4985-92d3-15b7c3e4fec6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:51.260619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 57/118] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:48:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:51.261681 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec8561-84d9-4877-9dcf-dd331eb042f9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:51.291576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cdade-a481-40f1-b0e5-73d323c622c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:51.292421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cdade-a481-40f1-b0e5-73d323c622c6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.292706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cdade-a481-40f1-b0e5-73d323c622c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.292985 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cdade-a481-40f1-b0e5-73d323c622c6 None admin] PATH_INFO: `/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.294167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cdade-a481-40f1-b0e5-73d323c622c6 None admin] RBAC: Authorizing `identity:get_user(user_id=7c47db9f01c84c63b57cf3ae5711ffb1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:51.295614 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cdade-a481-40f1-b0e5-73d323c622c6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:51.297298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 62/119] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:51 2022] GET /identity/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:51.298055 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cdade-a481-40f1-b0e5-73d323c622c6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:51.314977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:51.324482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:51.378817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec8561-84d9-4877-9dcf-dd331eb042f9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:51.408962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cdade-a481-40f1-b0e5-73d323c622c6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:51.413225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfe409-5ece-4f8e-8934-f480c0dad4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:51.414024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfe409-5ece-4f8e-8934-f480c0dad4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.414306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfe409-5ece-4f8e-8934-f480c0dad4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.414582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfe409-5ece-4f8e-8934-f480c0dad4b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.425013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfe409-5ece-4f8e-8934-f480c0dad4b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-fcbfe409-5ece-4f8e-8934-f480c0dad4b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:51.432590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:51.433415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:48:51.434186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 58/120] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:51.435251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbf432-ca85-4c42-be14-e82ff7d96a43 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.435535 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbf432-ca85-4c42-be14-e82ff7d96a43 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.435836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbf432-ca85-4c42-be14-e82ff7d96a43 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.437248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 59/121] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:51.441742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbf432-ca85-4c42-be14-e82ff7d96a43 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:51.452290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3d511-1978-4f15-9d06-955b9395e68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:51.452955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3d511-1978-4f15-9d06-955b9395e68d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.453245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3d511-1978-4f15-9d06-955b9395e68d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.453529 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3d511-1978-4f15-9d06-955b9395e68d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.454372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3d511-1978-4f15-9d06-955b9395e68d None admin] RBAC: Adding query filter params (role.id=34f334464e614ee784ee399c7a70719f, scope.project.id=2f8281ddbf1345088c2c4475cad30949, user.id=90372fe182f149819ca0ca377fc079b2) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:51.455217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3d511-1978-4f15-9d06-955b9395e68d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:51.456719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3d511-1978-4f15-9d06-955b9395e68d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:51.471989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 63/122] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:48:51 2022] GET /identity/v3/role_assignments?user.id=90372fe182f149819ca0ca377fc079b2&scope.project.id=2f8281ddbf1345088c2c4475cad30949&role.id=34f334464e614ee784ee399c7a70719f => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:51.471989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3d511-1978-4f15-9d06-955b9395e68d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:51.557304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:51.562583 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbf432-ca85-4c42-be14-e82ff7d96a43 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:51.600601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:51.615528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfeb635-2ff9-449e-ae53-06319160aaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:51.616740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfeb635-2ff9-449e-ae53-06319160aaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.617031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfeb635-2ff9-449e-ae53-06319160aaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.617360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfeb635-2ff9-449e-ae53-06319160aaf4 None admin] PATH_INFO: `/v3/users/95e96acf6a87449d9aa6f29de04b192f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.618645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfeb635-2ff9-449e-ae53-06319160aaf4 None admin] RBAC: Authorizing `identity:get_user(user_id=95e96acf6a87449d9aa6f29de04b192f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:51.618834 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3d511-1978-4f15-9d06-955b9395e68d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:51.620849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfeb635-2ff9-449e-ae53-06319160aaf4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:51.623126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 60/123] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:51 2022] GET /identity/v3/users/95e96acf6a87449d9aa6f29de04b192f => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:51.623970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfeb635-2ff9-449e-ae53-06319160aaf4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:51.660883 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e820da-3e91-4b80-a2be-10eb840d544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:51.662029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e820da-3e91-4b80-a2be-10eb840d544e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.662352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e820da-3e91-4b80-a2be-10eb840d544e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.662638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e820da-3e91-4b80-a2be-10eb840d544e None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.663903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e820da-3e91-4b80-a2be-10eb840d544e None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:51.665413 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e820da-3e91-4b80-a2be-10eb840d544e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:51.667547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 64/124] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:51 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:51.673273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f94e11-78bb-4823-8189-0e6d94ff0644 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.673599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f94e11-78bb-4823-8189-0e6d94ff0644 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.673883 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f94e11-78bb-4823-8189-0e6d94ff0644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.692843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-42f94e11-78bb-4823-8189-0e6d94ff0644 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:51.694064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-42f94e11-78bb-4823-8189-0e6d94ff0644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:51.755989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfeb635-2ff9-449e-ae53-06319160aaf4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:51.768971 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 65/125] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:51.769730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f94e11-78bb-4823-8189-0e6d94ff0644 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:51.806709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcffb4-5fae-462a-b1bb-f94a5aeef39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:51.807382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcffb4-5fae-462a-b1bb-f94a5aeef39f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.807610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcffb4-5fae-462a-b1bb-f94a5aeef39f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.807911 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcffb4-5fae-462a-b1bb-f94a5aeef39f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.808670 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcffb4-5fae-462a-b1bb-f94a5aeef39f None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:51.808984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcffb4-5fae-462a-b1bb-f94a5aeef39f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:51.810363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcffb4-5fae-462a-b1bb-f94a5aeef39f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:51.824996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 61/126] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:48:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:51.826097 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcffb4-5fae-462a-b1bb-f94a5aeef39f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:51.866872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:51.873077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:51.897497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f94e11-78bb-4823-8189-0e6d94ff0644 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:51.945536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bb11a-fc90-4c19-a090-84b37debe4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:51.947329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bb11a-fc90-4c19-a090-84b37debe4af None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.948090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bb11a-fc90-4c19-a090-84b37debe4af None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.948569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bb11a-fc90-4c19-a090-84b37debe4af None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.949649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcffb4-5fae-462a-b1bb-f94a5aeef39f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:51.950195 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bb11a-fc90-4c19-a090-84b37debe4af None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:51.953076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bb11a-fc90-4c19-a090-84b37debe4af None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:51.955038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 66/127] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:51 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:51.956018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bb11a-fc90-4c19-a090-84b37debe4af None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:51.987382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af67565-8ef2-4f5e-8214-310437a6f2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:51.988331 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af67565-8ef2-4f5e-8214-310437a6f2e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.988617 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af67565-8ef2-4f5e-8214-310437a6f2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.988915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af67565-8ef2-4f5e-8214-310437a6f2e7 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949/users/7c47db9f01c84c63b57cf3ae5711ffb1/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.990968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af67565-8ef2-4f5e-8214-310437a6f2e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f8281ddbf1345088c2c4475cad30949, user_id=7c47db9f01c84c63b57cf3ae5711ffb1, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:51.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af67565-8ef2-4f5e-8214-310437a6f2e7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:52.015883 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 62/128] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:48:51 2022] PUT /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949/users/7c47db9f01c84c63b57cf3ae5711ffb1/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:48:52.017032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af67565-8ef2-4f5e-8214-310437a6f2e7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.076623 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bb11a-fc90-4c19-a090-84b37debe4af None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.116114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a3256-88be-42d5-812b-6e521ac38506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:52.117255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a3256-88be-42d5-812b-6e521ac38506 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.117739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a3256-88be-42d5-812b-6e521ac38506 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.118207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a3256-88be-42d5-812b-6e521ac38506 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.130442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a3256-88be-42d5-812b-6e521ac38506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6c6a3256-88be-42d5-812b-6e521ac38506 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:52.139098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:52.140045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:52.140822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:52.140822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:52.140822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:52.140822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:52.140822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:52.140822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:52.140822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:52.140822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:52.140822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:52.140822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:52.140822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:48:52.140822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:48:52.144354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 67/129] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:48:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:52.145387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a3256-88be-42d5-812b-6e521ac38506 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.162596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:52.183447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:52.211523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af67565-8ef2-4f5e-8214-310437a6f2e7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.266983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a3256-88be-42d5-812b-6e521ac38506 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.284011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b1f8a-2af3-4b6d-9fc1-ef109eddf906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:52.285213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b1f8a-2af3-4b6d-9fc1-ef109eddf906 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.285644 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b1f8a-2af3-4b6d-9fc1-ef109eddf906 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.286066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b1f8a-2af3-4b6d-9fc1-ef109eddf906 None admin] PATH_INFO: `/v3/users/95e96acf6a87449d9aa6f29de04b192f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.287506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b1f8a-2af3-4b6d-9fc1-ef109eddf906 None admin] RBAC: Authorizing `identity:get_user(user_id=95e96acf6a87449d9aa6f29de04b192f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:52.289187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b1f8a-2af3-4b6d-9fc1-ef109eddf906 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:52.291083 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 63/130] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:52 2022] GET /identity/v3/users/95e96acf6a87449d9aa6f29de04b192f => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:52.295220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b1f8a-2af3-4b6d-9fc1-ef109eddf906 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.341191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464604b-a11a-4c17-a74c-f1c77ac027ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:52.342418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464604b-a11a-4c17-a74c-f1c77ac027ba None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.342913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464604b-a11a-4c17-a74c-f1c77ac027ba None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.343333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464604b-a11a-4c17-a74c-f1c77ac027ba None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949/users/95e96acf6a87449d9aa6f29de04b192f/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.345765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464604b-a11a-4c17-a74c-f1c77ac027ba None admin] RBAC: Authorizing `identity:create_grant(project_id=2f8281ddbf1345088c2c4475cad30949, user_id=95e96acf6a87449d9aa6f29de04b192f, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:52.347376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464604b-a11a-4c17-a74c-f1c77ac027ba None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:52.366307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 68/131] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:48:52 2022] PUT /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949/users/95e96acf6a87449d9aa6f29de04b192f/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:48:52.367505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464604b-a11a-4c17-a74c-f1c77ac027ba None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.424710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b1f8a-2af3-4b6d-9fc1-ef109eddf906 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.456747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:52.462414 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13b70d-05a6-45b8-a583-27ae84eeda3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:52.463163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13b70d-05a6-45b8-a583-27ae84eeda3d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.463458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13b70d-05a6-45b8-a583-27ae84eeda3d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.463813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13b70d-05a6-45b8-a583-27ae84eeda3d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.464587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13b70d-05a6-45b8-a583-27ae84eeda3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:52.464900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13b70d-05a6-45b8-a583-27ae84eeda3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:52.466551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13b70d-05a6-45b8-a583-27ae84eeda3d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:52.476327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 64/132] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:48:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:52.482082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13b70d-05a6-45b8-a583-27ae84eeda3d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.496481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464604b-a11a-4c17-a74c-f1c77ac027ba None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.504025 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:52.539635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554851f-8680-46b0-a328-4c7af59e03b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:52.540818 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554851f-8680-46b0-a328-4c7af59e03b4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.541383 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554851f-8680-46b0-a328-4c7af59e03b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.541809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554851f-8680-46b0-a328-4c7af59e03b4 None admin] PATH_INFO: `/v3/projects/cde95f4c105541eda1eb063175f7df78` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.543294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554851f-8680-46b0-a328-4c7af59e03b4 None admin] RBAC: Authorizing `identity:get_project(project_id=cde95f4c105541eda1eb063175f7df78)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:52.545256 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554851f-8680-46b0-a328-4c7af59e03b4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:52.547446 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 69/133] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:52 2022] GET /identity/v3/projects/cde95f4c105541eda1eb063175f7df78 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:52.553859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554851f-8680-46b0-a328-4c7af59e03b4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.595312 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13b70d-05a6-45b8-a583-27ae84eeda3d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.642809 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d3699-fe8c-42c8-8583-2a8c5da777b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:52.644390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d3699-fe8c-42c8-8583-2a8c5da777b3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.645586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d3699-fe8c-42c8-8583-2a8c5da777b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.646108 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d3699-fe8c-42c8-8583-2a8c5da777b3 None admin] PATH_INFO: `/v3/users/95e96acf6a87449d9aa6f29de04b192f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.648163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d3699-fe8c-42c8-8583-2a8c5da777b3 None admin] RBAC: Authorizing `identity:get_user(user_id=95e96acf6a87449d9aa6f29de04b192f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:52.650767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d3699-fe8c-42c8-8583-2a8c5da777b3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:52.652728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 65/134] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:52 2022] GET /identity/v3/users/95e96acf6a87449d9aa6f29de04b192f => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:52.659917 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d3699-fe8c-42c8-8583-2a8c5da777b3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.717568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554851f-8680-46b0-a328-4c7af59e03b4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.763148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056a027-8a42-4590-8f04-c8c92463f776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:52.764997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056a027-8a42-4590-8f04-c8c92463f776 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.765615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056a027-8a42-4590-8f04-c8c92463f776 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.766117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056a027-8a42-4590-8f04-c8c92463f776 None admin] PATH_INFO: `/v3/projects/cde95f4c105541eda1eb063175f7df78/users/95e96acf6a87449d9aa6f29de04b192f/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.768053 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:52.770231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056a027-8a42-4590-8f04-c8c92463f776 None admin] RBAC: Authorizing `identity:create_grant(project_id=cde95f4c105541eda1eb063175f7df78, user_id=95e96acf6a87449d9aa6f29de04b192f, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:52.772711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056a027-8a42-4590-8f04-c8c92463f776 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:52.787256 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:52.793095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 70/135] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:48:52 2022] PUT /identity/v3/projects/cde95f4c105541eda1eb063175f7df78/users/95e96acf6a87449d9aa6f29de04b192f/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:48:52.797141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d3699-fe8c-42c8-8583-2a8c5da777b3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.839000 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709dbba-e0b4-4136-a272-9654669c892e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:52.840268 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709dbba-e0b4-4136-a272-9654669c892e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.840833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709dbba-e0b4-4136-a272-9654669c892e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.841288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709dbba-e0b4-4136-a272-9654669c892e None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.842971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709dbba-e0b4-4136-a272-9654669c892e None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:52.844824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709dbba-e0b4-4136-a272-9654669c892e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:52.847145 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 66/136] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:52 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:52.853576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056a027-8a42-4590-8f04-c8c92463f776 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.988382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056a027-8a42-4590-8f04-c8c92463f776 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:53.030158 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22604a5-604c-4f2b-ba20-38f35d6a8ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:53.031349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22604a5-604c-4f2b-ba20-38f35d6a8ce0 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:53.031833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22604a5-604c-4f2b-ba20-38f35d6a8ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:53.032340 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22604a5-604c-4f2b-ba20-38f35d6a8ce0 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949/users/95e96acf6a87449d9aa6f29de04b192f/roles/8a0cfaebc67a497a98459da3f1a82ef9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:53.035339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22604a5-604c-4f2b-ba20-38f35d6a8ce0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f8281ddbf1345088c2c4475cad30949, user_id=95e96acf6a87449d9aa6f29de04b192f, role_id=8a0cfaebc67a497a98459da3f1a82ef9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:53.037024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22604a5-604c-4f2b-ba20-38f35d6a8ce0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:53.049958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691ded3-c01f-4f6f-a73e-cffb7ef55d68 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:53.050535 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691ded3-c01f-4f6f-a73e-cffb7ef55d68 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:53.050993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691ded3-c01f-4f6f-a73e-cffb7ef55d68 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:53.052969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 67/137] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:53.058538 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d08b7-4b43-4786-82aa-adaa838fc54b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:53.059057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d08b7-4b43-4786-82aa-adaa838fc54b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:53.059512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d08b7-4b43-4786-82aa-adaa838fc54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:53.078764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 71/138] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:48:52 2022] PUT /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949/users/95e96acf6a87449d9aa6f29de04b192f/roles/8a0cfaebc67a497a98459da3f1a82ef9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:48:53.088594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dc3d08b7-4b43-4786-82aa-adaa838fc54b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:53.090191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dc3d08b7-4b43-4786-82aa-adaa838fc54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:53.179315 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 68/139] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:53.205198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22604a5-604c-4f2b-ba20-38f35d6a8ce0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:53.280256 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:53.327693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22604a5-604c-4f2b-ba20-38f35d6a8ce0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:53.364229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4786e-3832-4170-87a8-89446011f883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:53.365356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4786e-3832-4170-87a8-89446011f883 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:53.365841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4786e-3832-4170-87a8-89446011f883 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:53.366343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4786e-3832-4170-87a8-89446011f883 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:53.367791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4786e-3832-4170-87a8-89446011f883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:53.369443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4786e-3832-4170-87a8-89446011f883 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:53.371707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 72/140] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:53.379584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d08b7-4b43-4786-82aa-adaa838fc54b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:53.496108 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:53.514014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d08b7-4b43-4786-82aa-adaa838fc54b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:53.550983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71a138-7598-40cf-ab8c-f5fa4b9fdb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:53.551688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71a138-7598-40cf-ab8c-f5fa4b9fdb23 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:53.551985 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71a138-7598-40cf-ab8c-f5fa4b9fdb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:53.552274 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71a138-7598-40cf-ab8c-f5fa4b9fdb23 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:53.553001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71a138-7598-40cf-ab8c-f5fa4b9fdb23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:53.554473 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71a138-7598-40cf-ab8c-f5fa4b9fdb23 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:53.582853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 69/141] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:48:53 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:54.317954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919b60c-8426-460c-8fce-2492ea8a9889 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:54.318449 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919b60c-8426-460c-8fce-2492ea8a9889 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:54.318551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919b60c-8426-460c-8fce-2492ea8a9889 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:54.320057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 73/142] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:54.325383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ee26-ac0b-4135-b1a6-dd6705d333f8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:54.325682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ee26-ac0b-4135-b1a6-dd6705d333f8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:54.325969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ee26-ac0b-4135-b1a6-dd6705d333f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:54.347065 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-57e0ee26-ac0b-4135-b1a6-dd6705d333f8 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:54.348219 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-57e0ee26-ac0b-4135-b1a6-dd6705d333f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:54.430049 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 70/143] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:54.468283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919b60c-8426-460c-8fce-2492ea8a9889 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:54.594026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919b60c-8426-460c-8fce-2492ea8a9889 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:54.624434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:54.633123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a14e29-a5bb-44ec-b8c2-0734fe9d9cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:54.634513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a14e29-a5bb-44ec-b8c2-0734fe9d9cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:54.634796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a14e29-a5bb-44ec-b8c2-0734fe9d9cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:54.635069 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a14e29-a5bb-44ec-b8c2-0734fe9d9cf0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:54.646519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a14e29-a5bb-44ec-b8c2-0734fe9d9cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-16a14e29-a5bb-44ec-b8c2-0734fe9d9cf0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:54.654398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:54.655242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:48:54.656049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 74/144] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:54.662168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0ee26-ac0b-4135-b1a6-dd6705d333f8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:54.760588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:54.802540 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0ee26-ac0b-4135-b1a6-dd6705d333f8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:54.841625 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2241d4b-9d3a-418e-b15e-04dc380791a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:54.842308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2241d4b-9d3a-418e-b15e-04dc380791a8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:54.842599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2241d4b-9d3a-418e-b15e-04dc380791a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:54.842884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2241d4b-9d3a-418e-b15e-04dc380791a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:54.843738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2241d4b-9d3a-418e-b15e-04dc380791a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:54.844055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2241d4b-9d3a-418e-b15e-04dc380791a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:54.845995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2241d4b-9d3a-418e-b15e-04dc380791a8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:54.855466 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 71/145] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:48:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:54.861635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a14e29-a5bb-44ec-b8c2-0734fe9d9cf0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:55.007387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a14e29-a5bb-44ec-b8c2-0734fe9d9cf0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:55.052231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f3120-03bb-406c-a49f-50709d896843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:55.053117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f3120-03bb-406c-a49f-50709d896843 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:55.053397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f3120-03bb-406c-a49f-50709d896843 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:55.053671 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f3120-03bb-406c-a49f-50709d896843 None admin] PATH_INFO: `/v3/users/90372fe182f149819ca0ca377fc079b2` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:55.054905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f3120-03bb-406c-a49f-50709d896843 None admin] RBAC: Authorizing `identity:get_user(user_id=90372fe182f149819ca0ca377fc079b2)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:55.056409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f3120-03bb-406c-a49f-50709d896843 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:55.058266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 75/146] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:54 2022] GET /identity/v3/users/90372fe182f149819ca0ca377fc079b2 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:55.068804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2241d4b-9d3a-418e-b15e-04dc380791a8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:55.212707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2241d4b-9d3a-418e-b15e-04dc380791a8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:55.248844 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:55.253880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e720e-8317-4d96-9d31-8027ca4c46d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:55.254969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e720e-8317-4d96-9d31-8027ca4c46d7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:55.255255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e720e-8317-4d96-9d31-8027ca4c46d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:55.255534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e720e-8317-4d96-9d31-8027ca4c46d7 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:55.256793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e720e-8317-4d96-9d31-8027ca4c46d7 None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:55.258329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e720e-8317-4d96-9d31-8027ca4c46d7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:55.260248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 72/147] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:55 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:55.265809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f3120-03bb-406c-a49f-50709d896843 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:55.356872 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:55.413695 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f3120-03bb-406c-a49f-50709d896843 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:55.456593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd035416-5078-49e1-96d5-d7470a976344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:55.457495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd035416-5078-49e1-96d5-d7470a976344 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:55.457787 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd035416-5078-49e1-96d5-d7470a976344 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:55.458061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd035416-5078-49e1-96d5-d7470a976344 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949/users/90372fe182f149819ca0ca377fc079b2/roles/34f334464e614ee784ee399c7a70719f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:55.460094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd035416-5078-49e1-96d5-d7470a976344 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f8281ddbf1345088c2c4475cad30949, user_id=90372fe182f149819ca0ca377fc079b2, role_id=34f334464e614ee784ee399c7a70719f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:55.461557 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd035416-5078-49e1-96d5-d7470a976344 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:55.486958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 76/148] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:48:55 2022] PUT /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949/users/90372fe182f149819ca0ca377fc079b2/roles/34f334464e614ee784ee399c7a70719f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:48:55.518977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95436bb-c0ca-44d5-9b27-056641bfda49 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:55.519313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95436bb-c0ca-44d5-9b27-056641bfda49 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:55.519545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95436bb-c0ca-44d5-9b27-056641bfda49 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:55.521142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 73/149] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:55.533730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad57555-e01a-446a-bc59-3848651cc8e6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:55.534030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad57555-e01a-446a-bc59-3848651cc8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:55.534298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad57555-e01a-446a-bc59-3848651cc8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:55.561011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2ad57555-e01a-446a-bc59-3848651cc8e6 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:55.562253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ad57555-e01a-446a-bc59-3848651cc8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:55.649046 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 77/150] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:55.680571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95436bb-c0ca-44d5-9b27-056641bfda49 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:55.807534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95436bb-c0ca-44d5-9b27-056641bfda49 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:55.842850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96198f-1c7a-476a-b389-8cc6a68a20fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:55.843749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96198f-1c7a-476a-b389-8cc6a68a20fa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:55.848057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96198f-1c7a-476a-b389-8cc6a68a20fa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:55.848374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96198f-1c7a-476a-b389-8cc6a68a20fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:55.860559 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96198f-1c7a-476a-b389-8cc6a68a20fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3b96198f-1c7a-476a-b389-8cc6a68a20fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:55.868442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:55.869348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:48:55.870142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 74/151] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:48:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:55.877140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad57555-e01a-446a-bc59-3848651cc8e6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:55.954914 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:55.984682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311c26-2511-48e6-9881-fc2f11792b26 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:55.985014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311c26-2511-48e6-9881-fc2f11792b26 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:55.985241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311c26-2511-48e6-9881-fc2f11792b26 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:55.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 75/152] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:55.992036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92934e8b-ba5c-4877-88c9-31941eed92db None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:55.992327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92934e8b-ba5c-4877-88c9-31941eed92db None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:55.992609 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92934e8b-ba5c-4877-88c9-31941eed92db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:56.015977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-92934e8b-ba5c-4877-88c9-31941eed92db None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:56.016615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad57555-e01a-446a-bc59-3848651cc8e6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:56.017751 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-92934e8b-ba5c-4877-88c9-31941eed92db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:56.101682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2b969-50f8-49f5-92af-082c1051f93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:56.102375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2b969-50f8-49f5-92af-082c1051f93d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:56.102662 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2b969-50f8-49f5-92af-082c1051f93d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:56.102939 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2b969-50f8-49f5-92af-082c1051f93d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:56.103730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2b969-50f8-49f5-92af-082c1051f93d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:56.104046 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2b969-50f8-49f5-92af-082c1051f93d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:56.105429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2b969-50f8-49f5-92af-082c1051f93d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:56.118868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 78/153] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:56.135558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:56.136196 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2b969-50f8-49f5-92af-082c1051f93d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:56.147954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 76/154] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:56.184212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92934e8b-ba5c-4877-88c9-31941eed92db None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:56.278781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2b969-50f8-49f5-92af-082c1051f93d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:56.298684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:56.349348 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc4da8-3beb-47d0-be9b-9a6d159e9076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:56.350232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc4da8-3beb-47d0-be9b-9a6d159e9076 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:56.350568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc4da8-3beb-47d0-be9b-9a6d159e9076 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:56.350796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc4da8-3beb-47d0-be9b-9a6d159e9076 None admin] PATH_INFO: `/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:56.352963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc4da8-3beb-47d0-be9b-9a6d159e9076 None admin] RBAC: Authorizing `identity:get_user(user_id=7c47db9f01c84c63b57cf3ae5711ffb1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:56.354451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc4da8-3beb-47d0-be9b-9a6d159e9076 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:56.356274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 79/155] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:56 2022] GET /identity/v3/users/7c47db9f01c84c63b57cf3ae5711ffb1 => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:56.357277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab754816-a462-4f47-bd6d-f42cddea5b4a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:56.357837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab754816-a462-4f47-bd6d-f42cddea5b4a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:56.357837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab754816-a462-4f47-bd6d-f42cddea5b4a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:56.359320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 80/156] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:56.363142 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab754816-a462-4f47-bd6d-f42cddea5b4a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:56.451974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92934e8b-ba5c-4877-88c9-31941eed92db None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:56.513336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab754816-a462-4f47-bd6d-f42cddea5b4a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:56.530331 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed811bc-5462-4159-979d-9d7fc724728a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:56.531532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed811bc-5462-4159-979d-9d7fc724728a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:56.532043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed811bc-5462-4159-979d-9d7fc724728a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:56.532674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed811bc-5462-4159-979d-9d7fc724728a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:56.544821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed811bc-5462-4159-979d-9d7fc724728a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4ed811bc-5462-4159-979d-9d7fc724728a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:56.554202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:56.555336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:56.556226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:56.556226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:56.556226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:56.556226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:56.556226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:56.556226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:56.556226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:56.556226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:56.556226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:56.556226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:56.556226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:56.556226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:48:56.559236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 77/157] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:48:56.560923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e98e45-8c08-40d4-8495-e8ab90ec1278 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:56.561423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e98e45-8c08-40d4-8495-e8ab90ec1278 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:56.561912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e98e45-8c08-40d4-8495-e8ab90ec1278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:56.567439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1170b6b-e804-4113-b63e-2a6e83a06542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:56.568214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1170b6b-e804-4113-b63e-2a6e83a06542 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:56.568504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1170b6b-e804-4113-b63e-2a6e83a06542 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:56.568797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1170b6b-e804-4113-b63e-2a6e83a06542 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:56.581532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-35e98e45-8c08-40d4-8495-e8ab90ec1278 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:56.583172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-35e98e45-8c08-40d4-8495-e8ab90ec1278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:56.635298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:56.672855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1170b6b-e804-4113-b63e-2a6e83a06542 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:56.674876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1170b6b-e804-4113-b63e-2a6e83a06542 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:56.675461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:56.731414 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 78/158] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:56.733080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e98e45-8c08-40d4-8495-e8ab90ec1278 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:56.819238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 81/159] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:48:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:48:56.820444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d8895-bc0e-4b00-967f-53f36b622c97 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:56.820735 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d8895-bc0e-4b00-967f-53f36b622c97 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:56.821012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d8895-bc0e-4b00-967f-53f36b622c97 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:56.822413 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 82/160] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:56.822980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d8895-bc0e-4b00-967f-53f36b622c97 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:56.872289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e98e45-8c08-40d4-8495-e8ab90ec1278 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:56.910940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dd70d-a75a-4470-af60-cef78648ec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:56.912126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dd70d-a75a-4470-af60-cef78648ec5d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:56.912630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dd70d-a75a-4470-af60-cef78648ec5d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:56.913112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dd70d-a75a-4470-af60-cef78648ec5d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:56.914061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dd70d-a75a-4470-af60-cef78648ec5d None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:56.914580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dd70d-a75a-4470-af60-cef78648ec5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:56.916197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dd70d-a75a-4470-af60-cef78648ec5d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:56.926983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 79/161] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:48:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:56.928190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dd70d-a75a-4470-af60-cef78648ec5d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:56.976551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:57.024315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:57.054677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d8895-bc0e-4b00-967f-53f36b622c97 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:57.065306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dd70d-a75a-4470-af60-cef78648ec5d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:57.092789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf97a5-ccc4-4e01-9017-6aabd6c82e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:57.093968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf97a5-ccc4-4e01-9017-6aabd6c82e97 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:57.094450 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf97a5-ccc4-4e01-9017-6aabd6c82e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:57.095474 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf97a5-ccc4-4e01-9017-6aabd6c82e97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:57.107048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf97a5-ccc4-4e01-9017-6aabd6c82e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9abf97a5-ccc4-4e01-9017-6aabd6c82e97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:57.116874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:57.117750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:57.118527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:57.118527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:57.118527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:57.118527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:57.118527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:57.118527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:57.118527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:57.118527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:57.118527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:57.118527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:57.118527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:48:57.118527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:48:57.120854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 83/162] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:48:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:57.122240 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13b8c0-4378-4a1a-9fba-7ac3f94d8cf5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:57.122734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13b8c0-4378-4a1a-9fba-7ac3f94d8cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:57.123208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13b8c0-4378-4a1a-9fba-7ac3f94d8cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:57.133727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5b861-6695-4b47-99a0-d861824ceadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:57.135001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5b861-6695-4b47-99a0-d861824ceadb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:57.135001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5b861-6695-4b47-99a0-d861824ceadb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:57.135185 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5b861-6695-4b47-99a0-d861824ceadb None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:57.140399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5b861-6695-4b47-99a0-d861824ceadb None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:57.142418 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5b861-6695-4b47-99a0-d861824ceadb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:57.144194 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 80/163] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:56 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:57.144960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5b861-6695-4b47-99a0-d861824ceadb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:57.148111 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7f13b8c0-4378-4a1a-9fba-7ac3f94d8cf5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:57.149520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7f13b8c0-4378-4a1a-9fba-7ac3f94d8cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:57.225789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 84/164] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:57.226933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13b8c0-4378-4a1a-9fba-7ac3f94d8cf5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:57.298848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:57.306580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5b861-6695-4b47-99a0-d861824ceadb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:57.327190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:57.340893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13b8c0-4378-4a1a-9fba-7ac3f94d8cf5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:57.381867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e6b3e-1c05-4a1a-bd10-0908d415fc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:57.383336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e6b3e-1c05-4a1a-bd10-0908d415fc55 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:57.383884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e6b3e-1c05-4a1a-bd10-0908d415fc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:57.384485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e6b3e-1c05-4a1a-bd10-0908d415fc55 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:57.385441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e6b3e-1c05-4a1a-bd10-0908d415fc55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:57.385955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e6b3e-1c05-4a1a-bd10-0908d415fc55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:57.387530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e6b3e-1c05-4a1a-bd10-0908d415fc55 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:57.397584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 85/165] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:48:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:57.398748 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e6b3e-1c05-4a1a-bd10-0908d415fc55 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:57.403496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea389b-4ecb-4362-a68d-7a215bf3ca05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:57.403496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea389b-4ecb-4362-a68d-7a215bf3ca05 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:57.403496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea389b-4ecb-4362-a68d-7a215bf3ca05 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:57.403496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea389b-4ecb-4362-a68d-7a215bf3ca05 None admin] PATH_INFO: `/v3/users/95e96acf6a87449d9aa6f29de04b192f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:57.404112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea389b-4ecb-4362-a68d-7a215bf3ca05 None admin] RBAC: Authorizing `identity:get_user(user_id=95e96acf6a87449d9aa6f29de04b192f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:57.405589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea389b-4ecb-4362-a68d-7a215bf3ca05 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:57.407223 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 81/166] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:57 2022] GET /identity/v3/users/95e96acf6a87449d9aa6f29de04b192f => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:57.409623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea389b-4ecb-4362-a68d-7a215bf3ca05 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:57.513588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e6b3e-1c05-4a1a-bd10-0908d415fc55 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:57.542367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:57.559995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0ca97-13e6-46c9-a929-8b692a69b0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:57.561136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0ca97-13e6-46c9-a929-8b692a69b0ed None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:57.561641 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0ca97-13e6-46c9-a929-8b692a69b0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:57.562121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0ca97-13e6-46c9-a929-8b692a69b0ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:57.563168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0ca97-13e6-46c9-a929-8b692a69b0ed None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=2f8281ddbf1345088c2c4475cad30949, user.id=7c47db9f01c84c63b57cf3ae5711ffb1) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:57.564247 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0ca97-13e6-46c9-a929-8b692a69b0ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:57.566168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0ca97-13e6-46c9-a929-8b692a69b0ed None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:57.577027 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 86/167] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:48:57 2022] GET /identity/v3/role_assignments?user.id=7c47db9f01c84c63b57cf3ae5711ffb1&scope.project.id=2f8281ddbf1345088c2c4475cad30949&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:57.578939 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0ca97-13e6-46c9-a929-8b692a69b0ed None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:57.618412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea389b-4ecb-4362-a68d-7a215bf3ca05 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:57.694744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82191e43-5f27-425d-8877-5498c3b880a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:57.696576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82191e43-5f27-425d-8877-5498c3b880a2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:57.700003 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82191e43-5f27-425d-8877-5498c3b880a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:57.700575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82191e43-5f27-425d-8877-5498c3b880a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:57.724802 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82191e43-5f27-425d-8877-5498c3b880a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-82191e43-5f27-425d-8877-5498c3b880a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:57.734381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:57.735836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:57.736729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:57.736729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:57.736729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:57.736729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:57.736729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:57.736729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:57.736729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:57.736729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:57.736729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:57.736729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:57.736729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:57.736729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:48:57.739313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 82/168] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:57.740428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82191e43-5f27-425d-8877-5498c3b880a2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:57.795146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:57.830673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0ca97-13e6-46c9-a929-8b692a69b0ed None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:57.899945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbafc06-42f4-4ef9-a390-091ecfc661ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:57.901795 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbafc06-42f4-4ef9-a390-091ecfc661ed None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:57.902838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbafc06-42f4-4ef9-a390-091ecfc661ed None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:57.903502 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbafc06-42f4-4ef9-a390-091ecfc661ed None admin] PATH_INFO: `/v3/users/95e96acf6a87449d9aa6f29de04b192f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:57.906091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82191e43-5f27-425d-8877-5498c3b880a2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:57.906938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbafc06-42f4-4ef9-a390-091ecfc661ed None admin] RBAC: Authorizing `identity:get_user(user_id=95e96acf6a87449d9aa6f29de04b192f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:57.909225 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbafc06-42f4-4ef9-a390-091ecfc661ed None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:57.912066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 87/169] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:57 2022] GET /identity/v3/users/95e96acf6a87449d9aa6f29de04b192f => generated 275 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:57.913865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbafc06-42f4-4ef9-a390-091ecfc661ed None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:57.976079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855a4f7-c79c-4387-952a-ea6c86c9fa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:57.978191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855a4f7-c79c-4387-952a-ea6c86c9fa43 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:57.980051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855a4f7-c79c-4387-952a-ea6c86c9fa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:57.980537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855a4f7-c79c-4387-952a-ea6c86c9fa43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.032740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:58.095116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855a4f7-c79c-4387-952a-ea6c86c9fa43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:58.097793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855a4f7-c79c-4387-952a-ea6c86c9fa43 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:58.118753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbafc06-42f4-4ef9-a390-091ecfc661ed None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:58.163171 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:58.190366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f94ec-33f7-485e-8108-fc80719de885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:58.191052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f94ec-33f7-485e-8108-fc80719de885 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:58.191338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f94ec-33f7-485e-8108-fc80719de885 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:58.191609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f94ec-33f7-485e-8108-fc80719de885 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.192398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f94ec-33f7-485e-8108-fc80719de885 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:58.192707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f94ec-33f7-485e-8108-fc80719de885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:58.194095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f94ec-33f7-485e-8108-fc80719de885 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:58.211614 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 88/170] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:48:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:58.212320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f94ec-33f7-485e-8108-fc80719de885 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:58.272336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 83/171] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:48:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:48:58.273685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0a0aa-fac9-4f6d-b81b-24a84deaac28 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:58.273974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0a0aa-fac9-4f6d-b81b-24a84deaac28 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:58.274251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0a0aa-fac9-4f6d-b81b-24a84deaac28 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.275541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 84/172] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:58.276272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0a0aa-fac9-4f6d-b81b-24a84deaac28 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:58.394928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f94ec-33f7-485e-8108-fc80719de885 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:58.420625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:58.441901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ddc26-a666-45b3-8c20-8ad8a9bc0917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:58.442571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ddc26-a666-45b3-8c20-8ad8a9bc0917 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:58.442856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ddc26-a666-45b3-8c20-8ad8a9bc0917 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:58.443132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ddc26-a666-45b3-8c20-8ad8a9bc0917 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.465920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0a0aa-fac9-4f6d-b81b-24a84deaac28 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:58.501844 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e456ffd-b733-47f8-a483-8cdec96324d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:58.502727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e456ffd-b733-47f8-a483-8cdec96324d8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:58.503008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e456ffd-b733-47f8-a483-8cdec96324d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:58.503280 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e456ffd-b733-47f8-a483-8cdec96324d8 None admin] PATH_INFO: `/v3/users/95e96acf6a87449d9aa6f29de04b192f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.504628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e456ffd-b733-47f8-a483-8cdec96324d8 None admin] RBAC: Authorizing `identity:get_user(user_id=95e96acf6a87449d9aa6f29de04b192f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:58.506187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e456ffd-b733-47f8-a483-8cdec96324d8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:58.507864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 85/173] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:58 2022] GET /identity/v3/users/95e96acf6a87449d9aa6f29de04b192f => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:58.508635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e456ffd-b733-47f8-a483-8cdec96324d8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:58.522488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:58.539813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ddc26-a666-45b3-8c20-8ad8a9bc0917 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:58.541288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ddc26-a666-45b3-8c20-8ad8a9bc0917 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:58.647373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 89/174] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:48:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:48:58.648636 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e45b3-7ba2-4cf3-95f7-eae372243b87 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:58.648932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e45b3-7ba2-4cf3-95f7-eae372243b87 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:58.649215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e45b3-7ba2-4cf3-95f7-eae372243b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.668360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-990e45b3-7ba2-4cf3-95f7-eae372243b87 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:58.669712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-990e45b3-7ba2-4cf3-95f7-eae372243b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:58.685369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e456ffd-b733-47f8-a483-8cdec96324d8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:58.704023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:58.727594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb12da5-d640-40b3-8bf3-1e71a57634f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:58.728951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb12da5-d640-40b3-8bf3-1e71a57634f9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:58.729243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb12da5-d640-40b3-8bf3-1e71a57634f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:58.729789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb12da5-d640-40b3-8bf3-1e71a57634f9 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.731246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb12da5-d640-40b3-8bf3-1e71a57634f9 None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:58.733135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb12da5-d640-40b3-8bf3-1e71a57634f9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:58.736151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 86/175] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:58 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:58.736959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb12da5-d640-40b3-8bf3-1e71a57634f9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:58.758105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 90/176] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:58.758954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e45b3-7ba2-4cf3-95f7-eae372243b87 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:58.830841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:58.858146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb12da5-d640-40b3-8bf3-1e71a57634f9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:58.868417 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e45b3-7ba2-4cf3-95f7-eae372243b87 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:58.893923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a51a37-58ba-4b23-b179-5457956ed2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:58.894576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a51a37-58ba-4b23-b179-5457956ed2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:58.894860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a51a37-58ba-4b23-b179-5457956ed2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:58.895139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a51a37-58ba-4b23-b179-5457956ed2a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.903968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b2465-45a7-4869-ad4e-cd473037db85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:58.904862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b2465-45a7-4869-ad4e-cd473037db85 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:58.905142 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b2465-45a7-4869-ad4e-cd473037db85 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:58.905444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b2465-45a7-4869-ad4e-cd473037db85 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.906635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b2465-45a7-4869-ad4e-cd473037db85 None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:58.908138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b2465-45a7-4869-ad4e-cd473037db85 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:58.909840 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 91/177] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:58 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:58.911012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cd605-07b5-402d-a7bd-1ef066e68e9b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:58.911293 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cd605-07b5-402d-a7bd-1ef066e68e9b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:58.911568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cd605-07b5-402d-a7bd-1ef066e68e9b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.912811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 92/178] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:58.915750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cd605-07b5-402d-a7bd-1ef066e68e9b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:58.937586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:58.966707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a51a37-58ba-4b23-b179-5457956ed2a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:58.968197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a51a37-58ba-4b23-b179-5457956ed2a7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:59.016559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cd605-07b5-402d-a7bd-1ef066e68e9b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.039672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.050179 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c727b-cff0-41f6-a75c-811a6ff68bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.050838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c727b-cff0-41f6-a75c-811a6ff68bbb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.051123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c727b-cff0-41f6-a75c-811a6ff68bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.051405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c727b-cff0-41f6-a75c-811a6ff68bbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.052274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c727b-cff0-41f6-a75c-811a6ff68bbb None admin] RBAC: Adding query filter params (role.id=7d15e336f8034a1fbdef48cfb7c4f717, scope.project.id=2f8281ddbf1345088c2c4475cad30949, user.id=95e96acf6a87449d9aa6f29de04b192f) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:59.052592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c727b-cff0-41f6-a75c-811a6ff68bbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:59.054026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c727b-cff0-41f6-a75c-811a6ff68bbb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:59.059917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 93/179] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:48:58 2022] GET /identity/v3/role_assignments?user.id=95e96acf6a87449d9aa6f29de04b192f&scope.project.id=2f8281ddbf1345088c2c4475cad30949&role.id=7d15e336f8034a1fbdef48cfb7c4f717 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:59.061121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1644cf2-e47d-45e5-9a2d-beea2073cdb5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.061410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1644cf2-e47d-45e5-9a2d-beea2073cdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.061688 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1644cf2-e47d-45e5-9a2d-beea2073cdb5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.062965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 94/180] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:59.063697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1644cf2-e47d-45e5-9a2d-beea2073cdb5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.084784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 87/181] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:48:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:48:59.084784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5a415-dc3d-4fd8-8653-a8196874ca11 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.084784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5a415-dc3d-4fd8-8653-a8196874ca11 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.084784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5a415-dc3d-4fd8-8653-a8196874ca11 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.084784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 88/182] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:59.084784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5a415-dc3d-4fd8-8653-a8196874ca11 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.171469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.193472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1644cf2-e47d-45e5-9a2d-beea2073cdb5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.202868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5a415-dc3d-4fd8-8653-a8196874ca11 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.241312 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35698cd5-060c-46c6-a463-343ccfa7cc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.242107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35698cd5-060c-46c6-a463-343ccfa7cc11 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.242386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35698cd5-060c-46c6-a463-343ccfa7cc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.242663 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35698cd5-060c-46c6-a463-343ccfa7cc11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.248160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9dea85-5779-4b2d-bf0b-ff0ea03bd62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.248909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9dea85-5779-4b2d-bf0b-ff0ea03bd62b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.249191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9dea85-5779-4b2d-bf0b-ff0ea03bd62b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.249471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9dea85-5779-4b2d-bf0b-ff0ea03bd62b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.250311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9dea85-5779-4b2d-bf0b-ff0ea03bd62b None admin] RBAC: Adding query filter params (role.id=8a0cfaebc67a497a98459da3f1a82ef9, scope.project.id=2f8281ddbf1345088c2c4475cad30949, user.id=95e96acf6a87449d9aa6f29de04b192f) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:59.250624 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9dea85-5779-4b2d-bf0b-ff0ea03bd62b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:59.252700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9dea85-5779-4b2d-bf0b-ff0ea03bd62b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:59.257977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35698cd5-060c-46c6-a463-343ccfa7cc11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:59.259681 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 89/183] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/role_assignments?user.id=95e96acf6a87449d9aa6f29de04b192f&scope.project.id=2f8281ddbf1345088c2c4475cad30949&role.id=8a0cfaebc67a497a98459da3f1a82ef9 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:59.262202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb04b9-eb6e-4821-9c5b-be9c7702a143 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.262202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb04b9-eb6e-4821-9c5b-be9c7702a143 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.262202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb04b9-eb6e-4821-9c5b-be9c7702a143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-35698cd5-060c-46c6-a463-343ccfa7cc11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:59.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:59.266864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:59.267646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:59.267646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:59.267646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:59.267646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:59.267646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:59.267646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:59.267646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:59.267646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:59.267646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:59.267646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:59.267646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:59.267646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:48:59.268288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 95/184] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:59.270394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46601f41-e183-4112-992b-ef62df655420 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.270680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46601f41-e183-4112-992b-ef62df655420 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.270958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46601f41-e183-4112-992b-ef62df655420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.280655 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2ccb04b9-eb6e-4821-9c5b-be9c7702a143 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:59.282038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2ccb04b9-eb6e-4821-9c5b-be9c7702a143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:59.294158 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-46601f41-e183-4112-992b-ef62df655420 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:59.295911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-46601f41-e183-4112-992b-ef62df655420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:59.322635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.365603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 90/185] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:59.366539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb04b9-eb6e-4821-9c5b-be9c7702a143 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.380744 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 96/186] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:59.382443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68876734-9bb0-4ade-80c1-7bfe8f1afbd2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.382940 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68876734-9bb0-4ade-80c1-7bfe8f1afbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.383432 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68876734-9bb0-4ade-80c1-7bfe8f1afbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.403476 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-68876734-9bb0-4ade-80c1-7bfe8f1afbd2 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:59.404974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-68876734-9bb0-4ade-80c1-7bfe8f1afbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:59.492041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb04b9-eb6e-4821-9c5b-be9c7702a143 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.497404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 97/187] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:59.498252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68876734-9bb0-4ade-80c1-7bfe8f1afbd2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.504862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.533571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22717285-9040-49b6-8feb-c96b714be821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.534439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22717285-9040-49b6-8feb-c96b714be821 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.534721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22717285-9040-49b6-8feb-c96b714be821 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.534997 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22717285-9040-49b6-8feb-c96b714be821 None admin] PATH_INFO: `/v3/projects/cde95f4c105541eda1eb063175f7df78` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.536214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22717285-9040-49b6-8feb-c96b714be821 None admin] RBAC: Authorizing `identity:get_project(project_id=cde95f4c105541eda1eb063175f7df78)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:59.537678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22717285-9040-49b6-8feb-c96b714be821 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:59.539375 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 91/188] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/projects/cde95f4c105541eda1eb063175f7df78 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:59.540174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22717285-9040-49b6-8feb-c96b714be821 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.577106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.601504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68876734-9bb0-4ade-80c1-7bfe8f1afbd2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.634556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe7f89-2a1d-4d06-8c60-e4face2d745d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.635218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe7f89-2a1d-4d06-8c60-e4face2d745d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.635496 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe7f89-2a1d-4d06-8c60-e4face2d745d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.635798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe7f89-2a1d-4d06-8c60-e4face2d745d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.636540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe7f89-2a1d-4d06-8c60-e4face2d745d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:59.636841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe7f89-2a1d-4d06-8c60-e4face2d745d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:59.638213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe7f89-2a1d-4d06-8c60-e4face2d745d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:59.640341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22717285-9040-49b6-8feb-c96b714be821 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.646488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 98/189] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:59.647268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe7f89-2a1d-4d06-8c60-e4face2d745d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.674479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306a883-a96d-439f-b027-bb7979e3b930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.675271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306a883-a96d-439f-b027-bb7979e3b930 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.675550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306a883-a96d-439f-b027-bb7979e3b930 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.675866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306a883-a96d-439f-b027-bb7979e3b930 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.677148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306a883-a96d-439f-b027-bb7979e3b930 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:59.678619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306a883-a96d-439f-b027-bb7979e3b930 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:59.680293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 92/190] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:59.685397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306a883-a96d-439f-b027-bb7979e3b930 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.711843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.748367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe7f89-2a1d-4d06-8c60-e4face2d745d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.781728 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dd75e-bec0-4330-8410-64c7e41c1190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.782592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dd75e-bec0-4330-8410-64c7e41c1190 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.782724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306a883-a96d-439f-b027-bb7979e3b930 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.783145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dd75e-bec0-4330-8410-64c7e41c1190 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.783145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dd75e-bec0-4330-8410-64c7e41c1190 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.784832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.793579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9dd75e-bec0-4330-8410-64c7e41c1190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-dd9dd75e-bec0-4330-8410-64c7e41c1190 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:59.801702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:59.802580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 99/191] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:59.808291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dd75e-bec0-4330-8410-64c7e41c1190 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.816659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9b8c2-ad6a-400a-be57-5c4aa7c00d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.817451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9b8c2-ad6a-400a-be57-5c4aa7c00d95 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.817730 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9b8c2-ad6a-400a-be57-5c4aa7c00d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.818004 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9b8c2-ad6a-400a-be57-5c4aa7c00d95 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.828193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9b8c2-ad6a-400a-be57-5c4aa7c00d95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e8d9b8c2-ad6a-400a-be57-5c4aa7c00d95 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:59.836394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:59.837317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:48:59.838089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 93/192] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:59.838748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9b8c2-ad6a-400a-be57-5c4aa7c00d95 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.903282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dd75e-bec0-4330-8410-64c7e41c1190 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.927500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.936783 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3db0f9-9c16-4bf2-9560-0e6c62354ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.937433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3db0f9-9c16-4bf2-9560-0e6c62354ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.937716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3db0f9-9c16-4bf2-9560-0e6c62354ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.937992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3db0f9-9c16-4bf2-9560-0e6c62354ff7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.938079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9b8c2-ad6a-400a-be57-5c4aa7c00d95 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.939320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3db0f9-9c16-4bf2-9560-0e6c62354ff7 None admin] RBAC: Adding query filter params (role.id=7d15e336f8034a1fbdef48cfb7c4f717, scope.project.id=cde95f4c105541eda1eb063175f7df78, user.id=95e96acf6a87449d9aa6f29de04b192f) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:59.939631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3db0f9-9c16-4bf2-9560-0e6c62354ff7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:59.941091 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3db0f9-9c16-4bf2-9560-0e6c62354ff7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:59.946796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 100/193] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/role_assignments?user.id=95e96acf6a87449d9aa6f29de04b192f&scope.project.id=cde95f4c105541eda1eb063175f7df78&role.id=7d15e336f8034a1fbdef48cfb7c4f717 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:59.947548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3db0f9-9c16-4bf2-9560-0e6c62354ff7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.985778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d697d03-ac6e-432b-a064-623ab8fa1107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.986669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d697d03-ac6e-432b-a064-623ab8fa1107 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.986955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d697d03-ac6e-432b-a064-623ab8fa1107 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.987233 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d697d03-ac6e-432b-a064-623ab8fa1107 None admin] PATH_INFO: `/v3/users/469ec98d8ecf46efbe0a63cf47ad8af9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.004574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d697d03-ac6e-432b-a064-623ab8fa1107 None admin] RBAC: Authorizing `identity:get_user(user_id=469ec98d8ecf46efbe0a63cf47ad8af9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.005980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d697d03-ac6e-432b-a064-623ab8fa1107 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.008210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 94/194] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/users/469ec98d8ecf46efbe0a63cf47ad8af9 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.008852 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d697d03-ac6e-432b-a064-623ab8fa1107 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.037498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.068504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3db0f9-9c16-4bf2-9560-0e6c62354ff7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.114352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6d5ab-50aa-42dd-9539-8e1cc8c57f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.115092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6d5ab-50aa-42dd-9539-8e1cc8c57f47 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.115374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6d5ab-50aa-42dd-9539-8e1cc8c57f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.115698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6d5ab-50aa-42dd-9539-8e1cc8c57f47 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.116855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6d5ab-50aa-42dd-9539-8e1cc8c57f47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.118313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6d5ab-50aa-42dd-9539-8e1cc8c57f47 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.129380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d697d03-ac6e-432b-a064-623ab8fa1107 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.150296 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 101/195] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:48:59 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:00.152654 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6d5ab-50aa-42dd-9539-8e1cc8c57f47 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.183705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe6dd2-b4fa-453b-8724-789d0ed80b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.184074 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe6dd2-b4fa-453b-8724-789d0ed80b12 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.184264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe6dd2-b4fa-453b-8724-789d0ed80b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.184675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe6dd2-b4fa-453b-8724-789d0ed80b12 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.185703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe6dd2-b4fa-453b-8724-789d0ed80b12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:00.186023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe6dd2-b4fa-453b-8724-789d0ed80b12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.187497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe6dd2-b4fa-453b-8724-789d0ed80b12 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.197189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 95/196] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.198124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe6dd2-b4fa-453b-8724-789d0ed80b12 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.218079 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.283223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6d5ab-50aa-42dd-9539-8e1cc8c57f47 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.303325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.324515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a055fe1-b129-4df2-81f4-cb203e0b11cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.325061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe6dd2-b4fa-453b-8724-789d0ed80b12 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.325228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a055fe1-b129-4df2-81f4-cb203e0b11cd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.325483 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a055fe1-b129-4df2-81f4-cb203e0b11cd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.325755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a055fe1-b129-4df2-81f4-cb203e0b11cd None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.326683 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a055fe1-b129-4df2-81f4-cb203e0b11cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:00.327614 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a055fe1-b129-4df2-81f4-cb203e0b11cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.328415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a055fe1-b129-4df2-81f4-cb203e0b11cd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.338378 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 102/197] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.339028 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a055fe1-b129-4df2-81f4-cb203e0b11cd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.363007 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80792295-8c28-4710-ad3f-068c4e1d2711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.363701 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80792295-8c28-4710-ad3f-068c4e1d2711 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.363991 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80792295-8c28-4710-ad3f-068c4e1d2711 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.364272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80792295-8c28-4710-ad3f-068c4e1d2711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.449101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80792295-8c28-4710-ad3f-068c4e1d2711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.450831 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80792295-8c28-4710-ad3f-068c4e1d2711 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.470725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a055fe1-b129-4df2-81f4-cb203e0b11cd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.480473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.513306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398612e4-6a42-4e00-8027-586fca3f018e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.514345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398612e4-6a42-4e00-8027-586fca3f018e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.514948 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398612e4-6a42-4e00-8027-586fca3f018e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.515231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398612e4-6a42-4e00-8027-586fca3f018e None admin] PATH_INFO: `/v3/users/469ec98d8ecf46efbe0a63cf47ad8af9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.516487 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398612e4-6a42-4e00-8027-586fca3f018e None admin] RBAC: Authorizing `identity:get_user(user_id=469ec98d8ecf46efbe0a63cf47ad8af9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.517935 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398612e4-6a42-4e00-8027-586fca3f018e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.519941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 103/198] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/users/469ec98d8ecf46efbe0a63cf47ad8af9 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.520723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398612e4-6a42-4e00-8027-586fca3f018e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.557058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.571952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 96/199] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:00.573654 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80792295-8c28-4710-ad3f-068c4e1d2711 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.635988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398612e4-6a42-4e00-8027-586fca3f018e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.683126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec6bba-1cb3-4c93-b76f-bc1953f0af66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.684949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec6bba-1cb3-4c93-b76f-bc1953f0af66 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.685431 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec6bba-1cb3-4c93-b76f-bc1953f0af66 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.686116 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec6bba-1cb3-4c93-b76f-bc1953f0af66 None admin] PATH_INFO: `/v3/users/90372fe182f149819ca0ca377fc079b2` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.688140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec6bba-1cb3-4c93-b76f-bc1953f0af66 None admin] RBAC: Authorizing `identity:get_user(user_id=90372fe182f149819ca0ca377fc079b2)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.690315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec6bba-1cb3-4c93-b76f-bc1953f0af66 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.692858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 104/200] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/users/90372fe182f149819ca0ca377fc079b2 => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.693804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec6bba-1cb3-4c93-b76f-bc1953f0af66 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.694124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80792295-8c28-4710-ad3f-068c4e1d2711 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.734052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ea68b-f6f2-477c-b1ee-644cb6ef042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.734719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ea68b-f6f2-477c-b1ee-644cb6ef042c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.735361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ea68b-f6f2-477c-b1ee-644cb6ef042c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.735467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ea68b-f6f2-477c-b1ee-644cb6ef042c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.754379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.792757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.808974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ea68b-f6f2-477c-b1ee-644cb6ef042c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.810389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ea68b-f6f2-477c-b1ee-644cb6ef042c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.824440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec6bba-1cb3-4c93-b76f-bc1953f0af66 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.867738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336f7c6-7e34-4270-aefc-6e49e9acb98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.868607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336f7c6-7e34-4270-aefc-6e49e9acb98d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.868890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336f7c6-7e34-4270-aefc-6e49e9acb98d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.869166 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336f7c6-7e34-4270-aefc-6e49e9acb98d None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.870510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336f7c6-7e34-4270-aefc-6e49e9acb98d None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.872075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336f7c6-7e34-4270-aefc-6e49e9acb98d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.873804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 105/201] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.874651 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336f7c6-7e34-4270-aefc-6e49e9acb98d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.941902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 97/202] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:00.942855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ea68b-f6f2-477c-b1ee-644cb6ef042c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.999631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336f7c6-7e34-4270-aefc-6e49e9acb98d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:01.013997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:01.037852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f9b9c-21a1-4edc-8f28-8ab55444b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:01.038571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f9b9c-21a1-4edc-8f28-8ab55444b80f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:01.038949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f9b9c-21a1-4edc-8f28-8ab55444b80f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:01.039177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f9b9c-21a1-4edc-8f28-8ab55444b80f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:01.040959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:01.062758 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ea68b-f6f2-477c-b1ee-644cb6ef042c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:01.111301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb5334-90f7-4538-93b7-aa7a97ddc3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:01.112501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5334-90f7-4538-93b7-aa7a97ddc3eb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:01.113129 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5334-90f7-4538-93b7-aa7a97ddc3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:01.113504 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5334-90f7-4538-93b7-aa7a97ddc3eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:01.115152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb5334-90f7-4538-93b7-aa7a97ddc3eb None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=73330eaa9cf649e283e878927d1e2da5, user.id=469ec98d8ecf46efbe0a63cf47ad8af9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:01.115915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb5334-90f7-4538-93b7-aa7a97ddc3eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.117463 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb5334-90f7-4538-93b7-aa7a97ddc3eb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.121562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f9b9c-21a1-4edc-8f28-8ab55444b80f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.123020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f9b9c-21a1-4edc-8f28-8ab55444b80f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.124572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 98/203] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/role_assignments?user.id=469ec98d8ecf46efbe0a63cf47ad8af9&scope.project.id=73330eaa9cf649e283e878927d1e2da5&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:01.125412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb5334-90f7-4538-93b7-aa7a97ddc3eb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:01.247255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb5334-90f7-4538-93b7-aa7a97ddc3eb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:01.289025 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510cfd8-538d-481e-9c2a-165916610579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:01.289959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510cfd8-538d-481e-9c2a-165916610579 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:01.290250 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510cfd8-538d-481e-9c2a-165916610579 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:01.290537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510cfd8-538d-481e-9c2a-165916610579 None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:01.292110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510cfd8-538d-481e-9c2a-165916610579 None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.293891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510cfd8-538d-481e-9c2a-165916610579 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.296012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 99/204] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:01 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:01.302371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510cfd8-538d-481e-9c2a-165916610579 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:01.344146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:01.399082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 106/205] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:01.400510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f9b9c-21a1-4edc-8f28-8ab55444b80f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:01.430232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:01.513995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510cfd8-538d-481e-9c2a-165916610579 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:01.574621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3360d5-7602-45e1-8eac-b5c02aeea21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:01.575283 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3360d5-7602-45e1-8eac-b5c02aeea21c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:01.575569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3360d5-7602-45e1-8eac-b5c02aeea21c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:01.575871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3360d5-7602-45e1-8eac-b5c02aeea21c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:01.576713 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3360d5-7602-45e1-8eac-b5c02aeea21c None admin] RBAC: Adding query filter params (role.id=8a0cfaebc67a497a98459da3f1a82ef9, scope.project.id=73330eaa9cf649e283e878927d1e2da5, user.id=469ec98d8ecf46efbe0a63cf47ad8af9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:01.577026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3360d5-7602-45e1-8eac-b5c02aeea21c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.578478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3360d5-7602-45e1-8eac-b5c02aeea21c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.593011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 100/206] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:01 2022] GET /identity/v3/role_assignments?user.id=469ec98d8ecf46efbe0a63cf47ad8af9&scope.project.id=73330eaa9cf649e283e878927d1e2da5&role.id=8a0cfaebc67a497a98459da3f1a82ef9 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:01.596884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f9b9c-21a1-4edc-8f28-8ab55444b80f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:01.655958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85d6f9-0642-4ac3-87b0-5c490e138281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:01.656900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85d6f9-0642-4ac3-87b0-5c490e138281 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:01.657786 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85d6f9-0642-4ac3-87b0-5c490e138281 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:01.658064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85d6f9-0642-4ac3-87b0-5c490e138281 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:01.659348 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85d6f9-0642-4ac3-87b0-5c490e138281 None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.660894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85d6f9-0642-4ac3-87b0-5c490e138281 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.663989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 107/207] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:01 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:01.668999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3360d5-7602-45e1-8eac-b5c02aeea21c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:01.760677 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:01.781197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3360d5-7602-45e1-8eac-b5c02aeea21c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:01.816282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05924fa-943a-49c2-a657-9ac94d0c8e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:01.817051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05924fa-943a-49c2-a657-9ac94d0c8e19 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:01.817215 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05924fa-943a-49c2-a657-9ac94d0c8e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:01.817488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05924fa-943a-49c2-a657-9ac94d0c8e19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:01.818327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05924fa-943a-49c2-a657-9ac94d0c8e19 None admin] RBAC: Adding query filter params (role.id=34f334464e614ee784ee399c7a70719f, scope.project.id=2f8281ddbf1345088c2c4475cad30949, user.id=90372fe182f149819ca0ca377fc079b2) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:01.818643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05924fa-943a-49c2-a657-9ac94d0c8e19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.820132 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05924fa-943a-49c2-a657-9ac94d0c8e19 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.826450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 101/208] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:01 2022] GET /identity/v3/role_assignments?user.id=90372fe182f149819ca0ca377fc079b2&scope.project.id=2f8281ddbf1345088c2c4475cad30949&role.id=34f334464e614ee784ee399c7a70719f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:02.428534 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2275ad-455e-4649-8a6a-9791f580c4bd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:02.428871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2275ad-455e-4649-8a6a-9791f580c4bd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:02.428993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2275ad-455e-4649-8a6a-9791f580c4bd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:02.430380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 108/209] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:02.435562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de021059-0ddd-4369-8ed4-ab41d26d0d38 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:02.435896 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de021059-0ddd-4369-8ed4-ab41d26d0d38 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:02.436347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de021059-0ddd-4369-8ed4-ab41d26d0d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:02.454094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-de021059-0ddd-4369-8ed4-ab41d26d0d38 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:02.455273 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-de021059-0ddd-4369-8ed4-ab41d26d0d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:02.531892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 102/210] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:02.555133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2275ad-455e-4649-8a6a-9791f580c4bd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:02.635873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:02.643472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f27d40-67dd-48f8-b293-abde9dda62dd None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:02.644615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f27d40-67dd-48f8-b293-abde9dda62dd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:02.645140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f27d40-67dd-48f8-b293-abde9dda62dd None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:02.648321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 103/211] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:02.653481 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37261491-1bd4-4454-8959-463b22dd23a9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:02.654016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37261491-1bd4-4454-8959-463b22dd23a9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:02.654508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37261491-1bd4-4454-8959-463b22dd23a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:02.661304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2275ad-455e-4649-8a6a-9791f580c4bd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:02.672348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-37261491-1bd4-4454-8959-463b22dd23a9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:02.673793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-37261491-1bd4-4454-8959-463b22dd23a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:02.699932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05101995-9a4f-466b-a85b-293a42943042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:02.701074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05101995-9a4f-466b-a85b-293a42943042 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:02.701563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05101995-9a4f-466b-a85b-293a42943042 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:02.702051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05101995-9a4f-466b-a85b-293a42943042 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:02.714124 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05101995-9a4f-466b-a85b-293a42943042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-05101995-9a4f-466b-a85b-293a42943042 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:02.722963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:02.723885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:02.724673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:02.724673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:02.724673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:02.724673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:02.724673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:02.724673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:02.724673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:02.724673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:02.724673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:02.724673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:02.724673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:02.724673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:02.727109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 109/212] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:02.732782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05101995-9a4f-466b-a85b-293a42943042 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:02.749643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 104/213] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:02.774355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37261491-1bd4-4454-8959-463b22dd23a9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:02.778439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:02.874947 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05101995-9a4f-466b-a85b-293a42943042 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:02.892455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37261491-1bd4-4454-8959-463b22dd23a9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:02.916287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443fb76-5cd5-4f7e-a1d3-ef7f95eb7761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:02.917260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443fb76-5cd5-4f7e-a1d3-ef7f95eb7761 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:02.917747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443fb76-5cd5-4f7e-a1d3-ef7f95eb7761 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:02.918199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443fb76-5cd5-4f7e-a1d3-ef7f95eb7761 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:02.919240 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443fb76-5cd5-4f7e-a1d3-ef7f95eb7761 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:02.919755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443fb76-5cd5-4f7e-a1d3-ef7f95eb7761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:02.921423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443fb76-5cd5-4f7e-a1d3-ef7f95eb7761 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:02.931444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 110/214] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:02.936998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443fb76-5cd5-4f7e-a1d3-ef7f95eb7761 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:02.939764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d50787-37e6-4095-985d-78a0ea716c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:02.940596 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d50787-37e6-4095-985d-78a0ea716c66 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:02.941259 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:02.941378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d50787-37e6-4095-985d-78a0ea716c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:02.941619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d50787-37e6-4095-985d-78a0ea716c66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:02.944206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d50787-37e6-4095-985d-78a0ea716c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:02.945958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d50787-37e6-4095-985d-78a0ea716c66 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:02.947669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 105/215] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:02.953039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d50787-37e6-4095-985d-78a0ea716c66 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:03.037530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:03.068015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d50787-37e6-4095-985d-78a0ea716c66 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:03.072316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443fb76-5cd5-4f7e-a1d3-ef7f95eb7761 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:03.110896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e4a7b-4532-4f4d-915b-d04769a99bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:03.111936 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e4a7b-4532-4f4d-915b-d04769a99bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.112309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e4a7b-4532-4f4d-915b-d04769a99bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.112694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e4a7b-4532-4f4d-915b-d04769a99bf7 None admin] PATH_INFO: `/v3/users/8ee4a92050584f8d9642d4310d7f7a86` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.114946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd210a46-e811-45c2-aa2e-28cf69072f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:03.116679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd210a46-e811-45c2-aa2e-28cf69072f9f None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.117223 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd210a46-e811-45c2-aa2e-28cf69072f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.117651 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd210a46-e811-45c2-aa2e-28cf69072f9f None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.118512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd210a46-e811-45c2-aa2e-28cf69072f9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:03.120140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd210a46-e811-45c2-aa2e-28cf69072f9f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:03.128115 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e4a7b-4532-4f4d-915b-d04769a99bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=8ee4a92050584f8d9642d4310d7f7a86)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:03.129755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e4a7b-4532-4f4d-915b-d04769a99bf7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:03.135733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 111/216] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:02 2022] GET /identity/v3/users/8ee4a92050584f8d9642d4310d7f7a86 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:03.137476 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e4a7b-4532-4f4d-915b-d04769a99bf7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:03.154283 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 106/217] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:02 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:49:03.216771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:03.266164 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e4a7b-4532-4f4d-915b-d04769a99bf7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:03.305158 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc2b0d-58c1-4ad1-8b24-cc01039cab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:03.306171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc2b0d-58c1-4ad1-8b24-cc01039cab31 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.306627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc2b0d-58c1-4ad1-8b24-cc01039cab31 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.307080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc2b0d-58c1-4ad1-8b24-cc01039cab31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.381340 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc2b0d-58c1-4ad1-8b24-cc01039cab31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:03.383119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc2b0d-58c1-4ad1-8b24-cc01039cab31 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:03.445268 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7abb34-c7c2-4cec-8c2a-381f7a282380 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.445453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:03.445938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7abb34-c7c2-4cec-8c2a-381f7a282380 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.445938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7abb34-c7c2-4cec-8c2a-381f7a282380 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.447267 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 107/218] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:03.452990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b864d9-bee6-43c5-b887-66699acf5ee9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.453415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b864d9-bee6-43c5-b887-66699acf5ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.453712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b864d9-bee6-43c5-b887-66699acf5ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.479794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-68b864d9-bee6-43c5-b887-66699acf5ee9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:03.480925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-68b864d9-bee6-43c5-b887-66699acf5ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:03.506196 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 112/219] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:03.512473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc2b0d-58c1-4ad1-8b24-cc01039cab31 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:03.576553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 108/220] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:03.600122 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b864d9-bee6-43c5-b887-66699acf5ee9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:03.641559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc2b0d-58c1-4ad1-8b24-cc01039cab31 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:03.666929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:03.694251 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab721ee-da2e-4611-a47d-e25252b16b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:03.695475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab721ee-da2e-4611-a47d-e25252b16b30 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.695988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab721ee-da2e-4611-a47d-e25252b16b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.696464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab721ee-da2e-4611-a47d-e25252b16b30 None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.699765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab721ee-da2e-4611-a47d-e25252b16b30 None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:03.701449 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab721ee-da2e-4611-a47d-e25252b16b30 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:03.705018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 113/221] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:03 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:03.711922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab721ee-da2e-4611-a47d-e25252b16b30 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:03.748472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b864d9-bee6-43c5-b887-66699acf5ee9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:03.760438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:03.812754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13da745-905c-4fb3-b8fd-161a6eda660e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:03.813592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13da745-905c-4fb3-b8fd-161a6eda660e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.813875 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13da745-905c-4fb3-b8fd-161a6eda660e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.814153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13da745-905c-4fb3-b8fd-161a6eda660e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.824879 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13da745-905c-4fb3-b8fd-161a6eda660e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a13da745-905c-4fb3-b8fd-161a6eda660e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:03.832628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:03.833536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:03.834337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 109/222] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:03.835485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0e390-ecb8-4b99-8f1a-72544cd2fec3 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.840420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0e390-ecb8-4b99-8f1a-72544cd2fec3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.840420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0e390-ecb8-4b99-8f1a-72544cd2fec3 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.841621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 110/223] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:03.842332 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0e390-ecb8-4b99-8f1a-72544cd2fec3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:03.868511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab721ee-da2e-4611-a47d-e25252b16b30 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:03.905970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca524c-57bd-45be-9b9d-7a4a62c52a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:03.906651 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca524c-57bd-45be-9b9d-7a4a62c52a00 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.907012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca524c-57bd-45be-9b9d-7a4a62c52a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.907835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca524c-57bd-45be-9b9d-7a4a62c52a00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.909243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca524c-57bd-45be-9b9d-7a4a62c52a00 None admin] RBAC: Adding query filter params (role.id=7d15e336f8034a1fbdef48cfb7c4f717, scope.project.id=73330eaa9cf649e283e878927d1e2da5, user.id=8ee4a92050584f8d9642d4310d7f7a86) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:03.909563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca524c-57bd-45be-9b9d-7a4a62c52a00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:03.911459 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca524c-57bd-45be-9b9d-7a4a62c52a00 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:03.919716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 114/224] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:03 2022] GET /identity/v3/role_assignments?user.id=8ee4a92050584f8d9642d4310d7f7a86&scope.project.id=73330eaa9cf649e283e878927d1e2da5&role.id=7d15e336f8034a1fbdef48cfb7c4f717 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:03.921149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d41fb8-6a47-47ec-a098-17c8fd5c0fa7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.921449 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d41fb8-6a47-47ec-a098-17c8fd5c0fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.921715 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d41fb8-6a47-47ec-a098-17c8fd5c0fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.948249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a9d41fb8-6a47-47ec-a098-17c8fd5c0fa7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:03.949564 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a9d41fb8-6a47-47ec-a098-17c8fd5c0fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:04.034044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0e390-ecb8-4b99-8f1a-72544cd2fec3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.063858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 115/225] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:04.077491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:04.100811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d41fb8-6a47-47ec-a098-17c8fd5c0fa7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:04.114148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3eccb-c7a9-40c3-9fa9-478be52c6ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.115155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3eccb-c7a9-40c3-9fa9-478be52c6ded None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.115638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3eccb-c7a9-40c3-9fa9-478be52c6ded None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.116300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3eccb-c7a9-40c3-9fa9-478be52c6ded None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.117364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3eccb-c7a9-40c3-9fa9-478be52c6ded None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:04.117870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3eccb-c7a9-40c3-9fa9-478be52c6ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:04.128063 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:04.128412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3eccb-c7a9-40c3-9fa9-478be52c6ded None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:04.140488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 111/226] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:04.146854 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3eccb-c7a9-40c3-9fa9-478be52c6ded None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:04.205934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d41fb8-6a47-47ec-a098-17c8fd5c0fa7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.257312 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae99a1-f773-4469-82c1-abf82692a8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.258136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae99a1-f773-4469-82c1-abf82692a8bd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.258420 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae99a1-f773-4469-82c1-abf82692a8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.258699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae99a1-f773-4469-82c1-abf82692a8bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.286528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3eccb-c7a9-40c3-9fa9-478be52c6ded None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.292605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae99a1-f773-4469-82c1-abf82692a8bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-eaae99a1-f773-4469-82c1-abf82692a8bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:04.308934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:04.309867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:04.310665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 116/227] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:04.318768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae99a1-f773-4469-82c1-abf82692a8bd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:04.321251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879a661-3f07-46c7-ad7b-cdf1ab052d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.322138 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879a661-3f07-46c7-ad7b-cdf1ab052d8e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.322422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879a661-3f07-46c7-ad7b-cdf1ab052d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.322703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879a661-3f07-46c7-ad7b-cdf1ab052d8e None admin] PATH_INFO: `/v3/users/469ec98d8ecf46efbe0a63cf47ad8af9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.323936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879a661-3f07-46c7-ad7b-cdf1ab052d8e None admin] RBAC: Authorizing `identity:get_user(user_id=469ec98d8ecf46efbe0a63cf47ad8af9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:04.325400 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879a661-3f07-46c7-ad7b-cdf1ab052d8e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:04.327213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 112/228] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:04 2022] GET /identity/v3/users/469ec98d8ecf46efbe0a63cf47ad8af9 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:04.336555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879a661-3f07-46c7-ad7b-cdf1ab052d8e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:04.415790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:04.416355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:04.436403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae99a1-f773-4469-82c1-abf82692a8bd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.473141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83940f-ad12-43dd-922d-ce2607a9370c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.473822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83940f-ad12-43dd-922d-ce2607a9370c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.474103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83940f-ad12-43dd-922d-ce2607a9370c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.474378 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83940f-ad12-43dd-922d-ce2607a9370c None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.475132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83940f-ad12-43dd-922d-ce2607a9370c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:04.475439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83940f-ad12-43dd-922d-ce2607a9370c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:04.476928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83940f-ad12-43dd-922d-ce2607a9370c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:04.486367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 117/229] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:04.491956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83940f-ad12-43dd-922d-ce2607a9370c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:04.522832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879a661-3f07-46c7-ad7b-cdf1ab052d8e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.597432 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcfee4-6480-49a9-8dff-4d29e5bbf0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.598229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcfee4-6480-49a9-8dff-4d29e5bbf0bc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.598513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcfee4-6480-49a9-8dff-4d29e5bbf0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.598787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcfee4-6480-49a9-8dff-4d29e5bbf0bc None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.599837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83940f-ad12-43dd-922d-ce2607a9370c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.601914 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcfee4-6480-49a9-8dff-4d29e5bbf0bc None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:04.603476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcfee4-6480-49a9-8dff-4d29e5bbf0bc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:04.605424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 113/230] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:04 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:04.612720 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcfee4-6480-49a9-8dff-4d29e5bbf0bc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:04.660777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcac85c-2d2d-412c-aab2-fb4bfe646cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.661662 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcac85c-2d2d-412c-aab2-fb4bfe646cba None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.661945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcac85c-2d2d-412c-aab2-fb4bfe646cba None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.662217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcac85c-2d2d-412c-aab2-fb4bfe646cba None admin] PATH_INFO: `/v3/users/469ec98d8ecf46efbe0a63cf47ad8af9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.664084 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcac85c-2d2d-412c-aab2-fb4bfe646cba None admin] RBAC: Authorizing `identity:get_user(user_id=469ec98d8ecf46efbe0a63cf47ad8af9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:04.665519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcac85c-2d2d-412c-aab2-fb4bfe646cba None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:04.667347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 118/231] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:04 2022] GET /identity/v3/users/469ec98d8ecf46efbe0a63cf47ad8af9 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:04.672879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcac85c-2d2d-412c-aab2-fb4bfe646cba None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:04.701077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:04.785813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcfee4-6480-49a9-8dff-4d29e5bbf0bc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.828021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:04.856360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5c335-743f-40ea-9924-ac86c9fce2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.857180 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5c335-743f-40ea-9924-ac86c9fce2b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.857463 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5c335-743f-40ea-9924-ac86c9fce2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.857738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5c335-743f-40ea-9924-ac86c9fce2b5 None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5/users/469ec98d8ecf46efbe0a63cf47ad8af9/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.860753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5c335-743f-40ea-9924-ac86c9fce2b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=73330eaa9cf649e283e878927d1e2da5, user_id=469ec98d8ecf46efbe0a63cf47ad8af9, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:04.863344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5c335-743f-40ea-9924-ac86c9fce2b5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:04.873827 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcac85c-2d2d-412c-aab2-fb4bfe646cba None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.878205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 114/232] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:04 2022] PUT /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5/users/469ec98d8ecf46efbe0a63cf47ad8af9/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:04.920903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc06c0-fbd2-4d98-874f-e699a03c3691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.921785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc06c0-fbd2-4d98-874f-e699a03c3691 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.922068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc06c0-fbd2-4d98-874f-e699a03c3691 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.922346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc06c0-fbd2-4d98-874f-e699a03c3691 None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.923643 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc06c0-fbd2-4d98-874f-e699a03c3691 None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:04.925248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc06c0-fbd2-4d98-874f-e699a03c3691 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:04.927436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 119/233] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:04 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:04.933105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5c335-743f-40ea-9924-ac86c9fce2b5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.082832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5c335-743f-40ea-9924-ac86c9fce2b5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:05.128044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b78ed4-40a3-4376-82b6-e01ffa0c55be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:05.128881 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b78ed4-40a3-4376-82b6-e01ffa0c55be None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.129168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b78ed4-40a3-4376-82b6-e01ffa0c55be None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.129450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b78ed4-40a3-4376-82b6-e01ffa0c55be None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5/users/469ec98d8ecf46efbe0a63cf47ad8af9/roles/8a0cfaebc67a497a98459da3f1a82ef9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.133403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b78ed4-40a3-4376-82b6-e01ffa0c55be None admin] RBAC: Authorizing `identity:create_grant(project_id=73330eaa9cf649e283e878927d1e2da5, user_id=469ec98d8ecf46efbe0a63cf47ad8af9, role_id=8a0cfaebc67a497a98459da3f1a82ef9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:05.134855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b78ed4-40a3-4376-82b6-e01ffa0c55be None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:05.155192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 115/234] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:04 2022] PUT /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5/users/469ec98d8ecf46efbe0a63cf47ad8af9/roles/8a0cfaebc67a497a98459da3f1a82ef9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:05.608474 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbf769-4b7a-4c2f-b2e7-4559c7b0a48c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.609112 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbf769-4b7a-4c2f-b2e7-4559c7b0a48c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.609292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbf769-4b7a-4c2f-b2e7-4559c7b0a48c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.610864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 120/235] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:05.616073 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee3e33-3d1e-4a93-902d-84e32d358c40 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.616369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee3e33-3d1e-4a93-902d-84e32d358c40 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.616652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee3e33-3d1e-4a93-902d-84e32d358c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.633874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-48ee3e33-3d1e-4a93-902d-84e32d358c40 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:05.635079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-48ee3e33-3d1e-4a93-902d-84e32d358c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:05.666735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:05.708269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 116/236] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:05.732351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbf769-4b7a-4c2f-b2e7-4559c7b0a48c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.848060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:05.876758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbf769-4b7a-4c2f-b2e7-4559c7b0a48c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:05.923170 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cd239-cbdc-46e5-a640-841faed0b4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:05.924015 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cd239-cbdc-46e5-a640-841faed0b4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.924357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cd239-cbdc-46e5-a640-841faed0b4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.924642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cd239-cbdc-46e5-a640-841faed0b4e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.934149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75c6e3-2990-4ee7-b88d-1dd688e92061 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.934432 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75c6e3-2990-4ee7-b88d-1dd688e92061 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.934710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75c6e3-2990-4ee7-b88d-1dd688e92061 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.936744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 117/237] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:05.943044 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cd239-cbdc-46e5-a640-841faed0b4e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:05.943910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f45b6-ba43-425c-b7f1-79b376fa9bcc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.944387 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f45b6-ba43-425c-b7f1-79b376fa9bcc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.944865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f45b6-ba43-425c-b7f1-79b376fa9bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a98cd239-cbdc-46e5-a640-841faed0b4e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:05.952400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:05.953287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:05.954624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:05.954624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:05.954624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:05.954624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:05.954624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:05.954624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:05.954624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:05.954624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:05.954624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:05.954624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:05.954624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:05.954624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:05.959819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 121/238] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:05.960690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cd239-cbdc-46e5-a640-841faed0b4e9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.967750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-766f45b6-ba43-425c-b7f1-79b376fa9bcc None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:05.970128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-766f45b6-ba43-425c-b7f1-79b376fa9bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:06.065174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 118/239] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:06.094114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f45b6-ba43-425c-b7f1-79b376fa9bcc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.162217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cd239-cbdc-46e5-a640-841faed0b4e9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.205062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbabea9-eda6-4cad-9f53-aa1da3ffef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.206204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbabea9-eda6-4cad-9f53-aa1da3ffef20 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.206697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbabea9-eda6-4cad-9f53-aa1da3ffef20 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.207168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbabea9-eda6-4cad-9f53-aa1da3ffef20 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.208214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbabea9-eda6-4cad-9f53-aa1da3ffef20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:06.208533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbabea9-eda6-4cad-9f53-aa1da3ffef20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.209926 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbabea9-eda6-4cad-9f53-aa1da3ffef20 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.219620 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 122/240] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:06.225232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbabea9-eda6-4cad-9f53-aa1da3ffef20 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.228169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:06.280621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:06.298076 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f45b6-ba43-425c-b7f1-79b376fa9bcc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.340174 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbabea9-eda6-4cad-9f53-aa1da3ffef20 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.375723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b55b44-741d-450b-97e1-10095e8d3fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.376061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0705a-05dd-4709-8a87-77ed8eb56b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.376598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b55b44-741d-450b-97e1-10095e8d3fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.376881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b55b44-741d-450b-97e1-10095e8d3fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.377158 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b55b44-741d-450b-97e1-10095e8d3fe5 None admin] PATH_INFO: `/v3/users/3a280523992140c9847411a56bb2b8f6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.377672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0705a-05dd-4709-8a87-77ed8eb56b61 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.378137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0705a-05dd-4709-8a87-77ed8eb56b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.378596 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0705a-05dd-4709-8a87-77ed8eb56b61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.379986 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0705a-05dd-4709-8a87-77ed8eb56b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.387695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0705a-05dd-4709-8a87-77ed8eb56b61 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.390022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 119/241] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:06.398230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b55b44-741d-450b-97e1-10095e8d3fe5 None admin] RBAC: Authorizing `identity:get_user(user_id=3a280523992140c9847411a56bb2b8f6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.398811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0705a-05dd-4709-8a87-77ed8eb56b61 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.399734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b55b44-741d-450b-97e1-10095e8d3fe5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.401804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 123/242] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:06 2022] GET /identity/v3/users/3a280523992140c9847411a56bb2b8f6 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:06.408278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b55b44-741d-450b-97e1-10095e8d3fe5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.469994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:06.512156 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b55b44-741d-450b-97e1-10095e8d3fe5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.517341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0705a-05dd-4709-8a87-77ed8eb56b61 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.546966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795ab9b-0c47-49f5-97d7-7e7ee4feb54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.547818 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795ab9b-0c47-49f5-97d7-7e7ee4feb54a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.548150 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795ab9b-0c47-49f5-97d7-7e7ee4feb54a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.548428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795ab9b-0c47-49f5-97d7-7e7ee4feb54a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.554032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa45b7-fe02-42c2-848a-a40e19e30c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.554944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa45b7-fe02-42c2-848a-a40e19e30c7e None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.555434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa45b7-fe02-42c2-848a-a40e19e30c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.555924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa45b7-fe02-42c2-848a-a40e19e30c7e None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.556926 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa45b7-fe02-42c2-848a-a40e19e30c7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.558554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa45b7-fe02-42c2-848a-a40e19e30c7e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.581101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 120/243] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:06 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:49:06.582941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a042b1-dc4b-4abe-a403-6a4240e5a593 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.583426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a042b1-dc4b-4abe-a403-6a4240e5a593 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.583959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a042b1-dc4b-4abe-a403-6a4240e5a593 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.585824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 121/244] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:06.597817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac543c6-12cc-41c3-8d4c-196ad78521b6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.598435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac543c6-12cc-41c3-8d4c-196ad78521b6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.598935 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac543c6-12cc-41c3-8d4c-196ad78521b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.629770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795ab9b-0c47-49f5-97d7-7e7ee4feb54a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.630094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1ac543c6-12cc-41c3-8d4c-196ad78521b6 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:06.631264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795ab9b-0c47-49f5-97d7-7e7ee4feb54a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.631978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1ac543c6-12cc-41c3-8d4c-196ad78521b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:06.663811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:06.729767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:06.780584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 122/245] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:06.784491 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 124/246] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:06.789896 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac543c6-12cc-41c3-8d4c-196ad78521b6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.804295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795ab9b-0c47-49f5-97d7-7e7ee4feb54a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.895013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac543c6-12cc-41c3-8d4c-196ad78521b6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.916810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795ab9b-0c47-49f5-97d7-7e7ee4feb54a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.930249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31357122-1ba6-4d53-8da1-7478870bd55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.931102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31357122-1ba6-4d53-8da1-7478870bd55f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.931385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31357122-1ba6-4d53-8da1-7478870bd55f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.931685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31357122-1ba6-4d53-8da1-7478870bd55f None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.933773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31357122-1ba6-4d53-8da1-7478870bd55f None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.935255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31357122-1ba6-4d53-8da1-7478870bd55f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.938709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 123/247] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:06 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:06.952685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31357122-1ba6-4d53-8da1-7478870bd55f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.959393 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53b9f7-afd1-4d69-8f6a-2c498a557172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.960230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53b9f7-afd1-4d69-8f6a-2c498a557172 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.960596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53b9f7-afd1-4d69-8f6a-2c498a557172 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.961067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53b9f7-afd1-4d69-8f6a-2c498a557172 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.974933 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:06.982185 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53b9f7-afd1-4d69-8f6a-2c498a557172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-da53b9f7-afd1-4d69-8f6a-2c498a557172 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:06.991561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:06.992444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:06.993245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:06.993245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:06.993245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:06.993245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:06.993245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:06.993245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:06.993245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:06.993245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:06.993245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:06.993245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:06.993245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:06.993245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:06.994523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 125/248] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:07.004871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53b9f7-afd1-4d69-8f6a-2c498a557172 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:07.030279 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:07.083459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31357122-1ba6-4d53-8da1-7478870bd55f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:07.131765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea6b9b-90c0-4f4f-b216-3a7076ba8e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:07.132445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea6b9b-90c0-4f4f-b216-3a7076ba8e46 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.132734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea6b9b-90c0-4f4f-b216-3a7076ba8e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.133014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea6b9b-90c0-4f4f-b216-3a7076ba8e46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.133860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea6b9b-90c0-4f4f-b216-3a7076ba8e46 None admin] RBAC: Adding query filter params (role.id=34f334464e614ee784ee399c7a70719f, scope.project.id=73330eaa9cf649e283e878927d1e2da5, user.id=3a280523992140c9847411a56bb2b8f6) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:07.134169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea6b9b-90c0-4f4f-b216-3a7076ba8e46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:07.135635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea6b9b-90c0-4f4f-b216-3a7076ba8e46 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:07.143745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 124/249] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:06 2022] GET /identity/v3/role_assignments?user.id=3a280523992140c9847411a56bb2b8f6&scope.project.id=73330eaa9cf649e283e878927d1e2da5&role.id=34f334464e614ee784ee399c7a70719f => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:07.158348 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53b9f7-afd1-4d69-8f6a-2c498a557172 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:07.202952 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c184f11-9688-41f8-9205-6d051b679aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:07.204226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c184f11-9688-41f8-9205-6d051b679aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.204773 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c184f11-9688-41f8-9205-6d051b679aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.205603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c184f11-9688-41f8-9205-6d051b679aa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.206618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c184f11-9688-41f8-9205-6d051b679aa4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:07.207129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c184f11-9688-41f8-9205-6d051b679aa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:07.208825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c184f11-9688-41f8-9205-6d051b679aa4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:07.220840 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 126/250] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:07.226123 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea6b9b-90c0-4f4f-b216-3a7076ba8e46 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:07.326811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:07.343729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea6b9b-90c0-4f4f-b216-3a7076ba8e46 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:07.380549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697ca0f-4f84-44ca-872d-a1c086f2d989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:07.381439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697ca0f-4f84-44ca-872d-a1c086f2d989 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.381728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697ca0f-4f84-44ca-872d-a1c086f2d989 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.382011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697ca0f-4f84-44ca-872d-a1c086f2d989 None admin] PATH_INFO: `/v3/users/8ee4a92050584f8d9642d4310d7f7a86` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.383237 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697ca0f-4f84-44ca-872d-a1c086f2d989 None admin] RBAC: Authorizing `identity:get_user(user_id=8ee4a92050584f8d9642d4310d7f7a86)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:07.384738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697ca0f-4f84-44ca-872d-a1c086f2d989 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:07.386706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 125/251] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:07 2022] GET /identity/v3/users/8ee4a92050584f8d9642d4310d7f7a86 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:07.392036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c184f11-9688-41f8-9205-6d051b679aa4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:07.484553 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:07.509386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c184f11-9688-41f8-9205-6d051b679aa4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:07.553088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045879a-d30a-4c84-8382-78366557e794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:07.553960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045879a-d30a-4c84-8382-78366557e794 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.554245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045879a-d30a-4c84-8382-78366557e794 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.554525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045879a-d30a-4c84-8382-78366557e794 None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.557188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045879a-d30a-4c84-8382-78366557e794 None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:07.558734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045879a-d30a-4c84-8382-78366557e794 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:07.560729 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 127/252] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:07 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:07.566549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697ca0f-4f84-44ca-872d-a1c086f2d989 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:07.677738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697ca0f-4f84-44ca-872d-a1c086f2d989 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:07.713093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d23ac-c4dc-44f0-aef7-9743d29f38bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:07.713996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d23ac-c4dc-44f0-aef7-9743d29f38bc None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.714280 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d23ac-c4dc-44f0-aef7-9743d29f38bc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.714556 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d23ac-c4dc-44f0-aef7-9743d29f38bc None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5/users/8ee4a92050584f8d9642d4310d7f7a86/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.716644 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d23ac-c4dc-44f0-aef7-9743d29f38bc None admin] RBAC: Authorizing `identity:create_grant(project_id=73330eaa9cf649e283e878927d1e2da5, user_id=8ee4a92050584f8d9642d4310d7f7a86, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:07.718164 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d23ac-c4dc-44f0-aef7-9743d29f38bc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:07.733829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 126/253] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:07 2022] PUT /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5/users/8ee4a92050584f8d9642d4310d7f7a86/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:07.783236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de45993-397b-4530-93e6-1ee832d103dc None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.783546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de45993-397b-4530-93e6-1ee832d103dc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.783861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de45993-397b-4530-93e6-1ee832d103dc None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.785295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 128/254] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:07.791180 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00093f4b-ea8c-4003-9ff1-6efdb04306f0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.791482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00093f4b-ea8c-4003-9ff1-6efdb04306f0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.791847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00093f4b-ea8c-4003-9ff1-6efdb04306f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.810573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-00093f4b-ea8c-4003-9ff1-6efdb04306f0 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:07.811849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-00093f4b-ea8c-4003-9ff1-6efdb04306f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:07.815261 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:07.892726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 127/255] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:07.918180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de45993-397b-4530-93e6-1ee832d103dc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:07.992844 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba06a82-8b3e-4580-aa8c-2ac6bee82748 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.993134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba06a82-8b3e-4580-aa8c-2ac6bee82748 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.993333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba06a82-8b3e-4580-aa8c-2ac6bee82748 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.995002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 128/256] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:08.001584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112bf63-a8ea-4675-80e0-343a6d08ccf4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:08.001875 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112bf63-a8ea-4675-80e0-343a6d08ccf4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:08.002160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112bf63-a8ea-4675-80e0-343a6d08ccf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:08.021634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a112bf63-a8ea-4675-80e0-343a6d08ccf4 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:08.023638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a112bf63-a8ea-4675-80e0-343a6d08ccf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:08.052467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de45993-397b-4530-93e6-1ee832d103dc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:08.114071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 129/257] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:08.119031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c9f59-24ca-4320-bcb3-e59ab96e0e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:08.119905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c9f59-24ca-4320-bcb3-e59ab96e0e9d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:08.120195 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c9f59-24ca-4320-bcb3-e59ab96e0e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:08.120477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c9f59-24ca-4320-bcb3-e59ab96e0e9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:08.141516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112bf63-a8ea-4675-80e0-343a6d08ccf4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:08.151542 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c9f59-24ca-4320-bcb3-e59ab96e0e9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-827c9f59-24ca-4320-bcb3-e59ab96e0e9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:08.159707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:08.160586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:08.161379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:08.161379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:08.161379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:08.161379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:08.161379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:08.161379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:08.161379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:08.161379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:08.161379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:08.161379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:08.161379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:08.161379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:08.169357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 129/258] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:08.175008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c9f59-24ca-4320-bcb3-e59ab96e0e9d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:08.195820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:08.268266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:08.327406 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112bf63-a8ea-4675-80e0-343a6d08ccf4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:08.388306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e5435-6acd-4426-9e79-cc84d2843282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:08.389479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e5435-6acd-4426-9e79-cc84d2843282 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:08.389974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e5435-6acd-4426-9e79-cc84d2843282 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:08.390853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e5435-6acd-4426-9e79-cc84d2843282 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:08.419899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e5435-6acd-4426-9e79-cc84d2843282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e26e5435-6acd-4426-9e79-cc84d2843282 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:08.429663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:08.430564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:08.431356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:08.431356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:08.431356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:08.431356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:08.431356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:08.431356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:08.431356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:08.431356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:08.431356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:08.431356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:08.431356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:08.431356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:08.432060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c9f59-24ca-4320-bcb3-e59ab96e0e9d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:08.434304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 130/259] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:08.441236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e5435-6acd-4426-9e79-cc84d2843282 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:08.540177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c94a1-d55d-4a08-a655-6ad835d135d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:08.540781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c94a1-d55d-4a08-a655-6ad835d135d6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:08.541067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c94a1-d55d-4a08-a655-6ad835d135d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:08.541348 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c94a1-d55d-4a08-a655-6ad835d135d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:08.542121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c94a1-d55d-4a08-a655-6ad835d135d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:08.542435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c94a1-d55d-4a08-a655-6ad835d135d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:08.543858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c94a1-d55d-4a08-a655-6ad835d135d6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:08.559304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 130/260] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:08.565762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c94a1-d55d-4a08-a655-6ad835d135d6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:08.617628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e5435-6acd-4426-9e79-cc84d2843282 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:08.677631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:08.706114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c94a1-d55d-4a08-a655-6ad835d135d6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:08.728014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:08.739417 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec4577-b92d-499f-a4e4-4a71c875d5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:08.740137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec4577-b92d-499f-a4e4-4a71c875d5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:08.740424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec4577-b92d-499f-a4e4-4a71c875d5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:08.740708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec4577-b92d-499f-a4e4-4a71c875d5c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:08.741472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec4577-b92d-499f-a4e4-4a71c875d5c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:08.741783 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec4577-b92d-499f-a4e4-4a71c875d5c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:08.743195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec4577-b92d-499f-a4e4-4a71c875d5c6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:08.765700 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6d748-eb06-4345-8d64-c6abaeedb4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:08.766610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6d748-eb06-4345-8d64-c6abaeedb4ce None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:08.766893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6d748-eb06-4345-8d64-c6abaeedb4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:08.767171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6d748-eb06-4345-8d64-c6abaeedb4ce None admin] PATH_INFO: `/v3/users/469ec98d8ecf46efbe0a63cf47ad8af9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:08.767758 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 131/261] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:08.768405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6d748-eb06-4345-8d64-c6abaeedb4ce None admin] RBAC: Authorizing `identity:get_user(user_id=469ec98d8ecf46efbe0a63cf47ad8af9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:08.770288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec4577-b92d-499f-a4e4-4a71c875d5c6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:08.771361 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6d748-eb06-4345-8d64-c6abaeedb4ce None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:08.773182 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 131/262] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:08 2022] GET /identity/v3/users/469ec98d8ecf46efbe0a63cf47ad8af9 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:08.783596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6d748-eb06-4345-8d64-c6abaeedb4ce None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:08.903389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6d748-eb06-4345-8d64-c6abaeedb4ce None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:08.943180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324028e-a031-4725-b02e-1470a93c2516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:08.944513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324028e-a031-4725-b02e-1470a93c2516 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:08.944513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324028e-a031-4725-b02e-1470a93c2516 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:08.944676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324028e-a031-4725-b02e-1470a93c2516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:08.955952 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:09.006187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec4577-b92d-499f-a4e4-4a71c875d5c6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:09.026541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324028e-a031-4725-b02e-1470a93c2516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:09.032752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324028e-a031-4725-b02e-1470a93c2516 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:09.148349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7ed4c-2e53-426f-bced-8c354de222be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:09.148876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 132/263] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:09.150007 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7ed4c-2e53-426f-bced-8c354de222be None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.150509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7ed4c-2e53-426f-bced-8c354de222be None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.150972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7ed4c-2e53-426f-bced-8c354de222be None admin] PATH_INFO: `/v3/users/469ec98d8ecf46efbe0a63cf47ad8af9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.154112 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324028e-a031-4725-b02e-1470a93c2516 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:09.154979 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7ed4c-2e53-426f-bced-8c354de222be None admin] RBAC: Authorizing `identity:get_user(user_id=469ec98d8ecf46efbe0a63cf47ad8af9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:09.157296 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7ed4c-2e53-426f-bced-8c354de222be None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:09.159419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 132/264] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:08 2022] GET /identity/v3/users/469ec98d8ecf46efbe0a63cf47ad8af9 => generated 283 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:09.164832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7ed4c-2e53-426f-bced-8c354de222be None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:09.198642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:09.260310 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324028e-a031-4725-b02e-1470a93c2516 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:09.312039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:09.324520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa33b8-56e7-459d-bb58-7fc661494713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:09.325394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa33b8-56e7-459d-bb58-7fc661494713 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.325675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa33b8-56e7-459d-bb58-7fc661494713 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.325951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa33b8-56e7-459d-bb58-7fc661494713 None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.328115 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa33b8-56e7-459d-bb58-7fc661494713 None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:09.330137 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa33b8-56e7-459d-bb58-7fc661494713 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:09.332059 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 133/265] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:09 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:09.344982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa33b8-56e7-459d-bb58-7fc661494713 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:09.396388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7ed4c-2e53-426f-bced-8c354de222be None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:09.460493 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68223238-1b35-40b0-b86d-791cc137d715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:09.461077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68223238-1b35-40b0-b86d-791cc137d715 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.461358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68223238-1b35-40b0-b86d-791cc137d715 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.461694 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68223238-1b35-40b0-b86d-791cc137d715 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.481257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa33b8-56e7-459d-bb58-7fc661494713 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:09.491555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:09.524025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cac8a-7273-4da4-962e-b809135eb5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:09.525387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cac8a-7273-4da4-962e-b809135eb5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.525667 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cac8a-7273-4da4-962e-b809135eb5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.526150 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cac8a-7273-4da4-962e-b809135eb5b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.526984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cac8a-7273-4da4-962e-b809135eb5b4 None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=73330eaa9cf649e283e878927d1e2da5, user.id=469ec98d8ecf46efbe0a63cf47ad8af9) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:09.527294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cac8a-7273-4da4-962e-b809135eb5b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:09.529858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cac8a-7273-4da4-962e-b809135eb5b4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:09.542986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 134/266] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:09 2022] GET /identity/v3/role_assignments?user.id=469ec98d8ecf46efbe0a63cf47ad8af9&scope.project.id=73330eaa9cf649e283e878927d1e2da5&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:09.545095 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68223238-1b35-40b0-b86d-791cc137d715 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:09.546619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68223238-1b35-40b0-b86d-791cc137d715 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:09.638851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:09.706850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 133/267] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:09.712604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cac8a-7273-4da4-962e-b809135eb5b4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:09.822877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cac8a-7273-4da4-962e-b809135eb5b4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:09.858062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9fd92-ffd5-4e1c-8258-7036d2bf798b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:09.859192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9fd92-ffd5-4e1c-8258-7036d2bf798b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.859469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9fd92-ffd5-4e1c-8258-7036d2bf798b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.860399 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9fd92-ffd5-4e1c-8258-7036d2bf798b None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.861575 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9fd92-ffd5-4e1c-8258-7036d2bf798b None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:09.863020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9fd92-ffd5-4e1c-8258-7036d2bf798b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:09.865037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 135/268] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:09 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:09.871070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68223238-1b35-40b0-b86d-791cc137d715 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:09.973736 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68223238-1b35-40b0-b86d-791cc137d715 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.008975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1c4a7-9a7d-4429-859c-94fbcbdd8f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.009635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1c4a7-9a7d-4429-859c-94fbcbdd8f34 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.009912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1c4a7-9a7d-4429-859c-94fbcbdd8f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.010186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1c4a7-9a7d-4429-859c-94fbcbdd8f34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.011014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1c4a7-9a7d-4429-859c-94fbcbdd8f34 None admin] RBAC: Adding query filter params (role.id=8a0cfaebc67a497a98459da3f1a82ef9, scope.project.id=73330eaa9cf649e283e878927d1e2da5, user.id=469ec98d8ecf46efbe0a63cf47ad8af9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:10.011322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1c4a7-9a7d-4429-859c-94fbcbdd8f34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.012796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1c4a7-9a7d-4429-859c-94fbcbdd8f34 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.019370 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 134/269] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:09 2022] GET /identity/v3/role_assignments?user.id=469ec98d8ecf46efbe0a63cf47ad8af9&scope.project.id=73330eaa9cf649e283e878927d1e2da5&role.id=8a0cfaebc67a497a98459da3f1a82ef9 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:10.085125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d84b7-ec7e-435f-9dd0-e268a426f81d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.085634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d84b7-ec7e-435f-9dd0-e268a426f81d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.085708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d84b7-ec7e-435f-9dd0-e268a426f81d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.090452 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 136/270] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:10.101354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afe486-4ff6-491a-9ead-e3b40604760c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.101646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afe486-4ff6-491a-9ead-e3b40604760c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.101924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afe486-4ff6-491a-9ead-e3b40604760c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.115072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c5000-39ed-4944-b799-dd2a11380fe6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.115355 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c5000-39ed-4944-b799-dd2a11380fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.115612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c5000-39ed-4944-b799-dd2a11380fe6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.117137 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 137/271] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:10.122611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daab0c-9a9b-4f6e-9105-92ce13a8196f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.123201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daab0c-9a9b-4f6e-9105-92ce13a8196f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.123778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daab0c-9a9b-4f6e-9105-92ce13a8196f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.127177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c9afe486-4ff6-491a-9ead-e3b40604760c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:10.128500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c9afe486-4ff6-491a-9ead-e3b40604760c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:10.143162 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-91daab0c-9a9b-4f6e-9105-92ce13a8196f None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:10.144756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-91daab0c-9a9b-4f6e-9105-92ce13a8196f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:10.164965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.223490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.258220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 138/272] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:10.263458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 135/273] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:10.282030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daab0c-9a9b-4f6e-9105-92ce13a8196f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.289985 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afe486-4ff6-491a-9ead-e3b40604760c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.394193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daab0c-9a9b-4f6e-9105-92ce13a8196f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.410361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afe486-4ff6-491a-9ead-e3b40604760c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.429970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa284e90-0d07-46f8-9b7b-dda08909717e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.430775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa284e90-0d07-46f8-9b7b-dda08909717e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.431067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa284e90-0d07-46f8-9b7b-dda08909717e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.431351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa284e90-0d07-46f8-9b7b-dda08909717e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.442854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa284e90-0d07-46f8-9b7b-dda08909717e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:10.446569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c5356-0f37-4d2f-a934-9400a8c2a62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.447368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c5356-0f37-4d2f-a934-9400a8c2a62a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.447646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c5356-0f37-4d2f-a934-9400a8c2a62a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.447945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c5356-0f37-4d2f-a934-9400a8c2a62a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.449114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c5356-0f37-4d2f-a934-9400a8c2a62a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.450561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c5356-0f37-4d2f-a934-9400a8c2a62a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-aa284e90-0d07-46f8-9b7b-dda08909717e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:10.451294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:10.452213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:10.454390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:10.454390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:10.454390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:10.454390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:10.454390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:10.454390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:10.454390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:10.454390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:10.454390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:10.454390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:10.454390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:10.454390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:10.455861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 136/274] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:10.456080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 139/275] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:10.458419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c5356-0f37-4d2f-a934-9400a8c2a62a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.459626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa284e90-0d07-46f8-9b7b-dda08909717e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.494259 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.505294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.598490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa284e90-0d07-46f8-9b7b-dda08909717e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.599848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c5356-0f37-4d2f-a934-9400a8c2a62a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.644161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5e5d7-e08e-4e3f-9a3d-47214eb03c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.644996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5e5d7-e08e-4e3f-9a3d-47214eb03c98 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.645371 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5e5d7-e08e-4e3f-9a3d-47214eb03c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.645843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5e5d7-e08e-4e3f-9a3d-47214eb03c98 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.646786 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5e5d7-e08e-4e3f-9a3d-47214eb03c98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:10.647190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5e5d7-e08e-4e3f-9a3d-47214eb03c98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.647681 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c77851-a412-4b65-9983-363c69a244fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.648516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c77851-a412-4b65-9983-363c69a244fe None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.648869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5e5d7-e08e-4e3f-9a3d-47214eb03c98 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.649112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c77851-a412-4b65-9983-363c69a244fe None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.649532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c77851-a412-4b65-9983-363c69a244fe None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.650688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c77851-a412-4b65-9983-363c69a244fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.652743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c77851-a412-4b65-9983-363c69a244fe None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.658462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 140/276] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:10.664480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5e5d7-e08e-4e3f-9a3d-47214eb03c98 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.680745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 137/277] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:10 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:10.713051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a350978-2d16-4643-8387-32235603cb8a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.713342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a350978-2d16-4643-8387-32235603cb8a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.713622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a350978-2d16-4643-8387-32235603cb8a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.715357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 138/278] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:10.721168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bd22d-c518-4ec0-8e9b-002dd1835559 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.721461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bd22d-c518-4ec0-8e9b-002dd1835559 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.721734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bd22d-c518-4ec0-8e9b-002dd1835559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.740604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fa6bd22d-c518-4ec0-8e9b-002dd1835559 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:10.741924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fa6bd22d-c518-4ec0-8e9b-002dd1835559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:10.779735 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.791245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5e5d7-e08e-4e3f-9a3d-47214eb03c98 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.828170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 139/279] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:10.829926 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b294003-82ab-4cec-9289-5748a24127de None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.830221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b294003-82ab-4cec-9289-5748a24127de None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.830503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b294003-82ab-4cec-9289-5748a24127de None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.832013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 140/280] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:10.837097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21276c78-6f73-45d8-b72e-af881a0e98ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.837277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef76975-1474-4bde-9246-b38263abdcf7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.837572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef76975-1474-4bde-9246-b38263abdcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.837863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef76975-1474-4bde-9246-b38263abdcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.838821 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21276c78-6f73-45d8-b72e-af881a0e98ad None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.839298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21276c78-6f73-45d8-b72e-af881a0e98ad None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.839911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21276c78-6f73-45d8-b72e-af881a0e98ad None admin] PATH_INFO: `/v3/users/3a280523992140c9847411a56bb2b8f6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.841551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21276c78-6f73-45d8-b72e-af881a0e98ad None admin] RBAC: Authorizing `identity:get_user(user_id=3a280523992140c9847411a56bb2b8f6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.843506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21276c78-6f73-45d8-b72e-af881a0e98ad None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.846424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 141/281] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/users/3a280523992140c9847411a56bb2b8f6 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:10.852567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21276c78-6f73-45d8-b72e-af881a0e98ad None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.857171 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3ef76975-1474-4bde-9246-b38263abdcf7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:10.858373 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3ef76975-1474-4bde-9246-b38263abdcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:10.861534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.939516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 141/282] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:10.940791 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef76975-1474-4bde-9246-b38263abdcf7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.977292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21276c78-6f73-45d8-b72e-af881a0e98ad None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.020685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301acdc3-32f2-4851-abb8-4049c6b21df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.023718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301acdc3-32f2-4851-abb8-4049c6b21df0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.023718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301acdc3-32f2-4851-abb8-4049c6b21df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.023718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301acdc3-32f2-4851-abb8-4049c6b21df0 None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.027484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301acdc3-32f2-4851-abb8-4049c6b21df0 None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.028997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301acdc3-32f2-4851-abb8-4049c6b21df0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.032023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 142/283] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:11.034277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301acdc3-32f2-4851-abb8-4049c6b21df0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.064445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef76975-1474-4bde-9246-b38263abdcf7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.075204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:11.115301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc16675-8801-4002-9aa7-c314fd62e3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.116788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc16675-8801-4002-9aa7-c314fd62e3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.117513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc16675-8801-4002-9aa7-c314fd62e3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.117991 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc16675-8801-4002-9aa7-c314fd62e3b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.133819 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc16675-8801-4002-9aa7-c314fd62e3b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7bc16675-8801-4002-9aa7-c314fd62e3b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:11.144286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:11.145181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:11.145966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:11.145966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:11.145966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:11.145966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:11.145966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:11.145966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:11.145966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:11.145966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:11.145966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:11.145966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:11.145966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:11.145966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:11.148625 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 142/284] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:11.149572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc16675-8801-4002-9aa7-c314fd62e3b3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.155878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:11.168191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301acdc3-32f2-4851-abb8-4049c6b21df0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.204307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f1c73-d9b9-4780-93d3-87d81d608ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.205261 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f1c73-d9b9-4780-93d3-87d81d608ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.205627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f1c73-d9b9-4780-93d3-87d81d608ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.205994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f1c73-d9b9-4780-93d3-87d81d608ca9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.221242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f1c73-d9b9-4780-93d3-87d81d608ca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-909f1c73-d9b9-4780-93d3-87d81d608ca9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:11.232295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:11.233706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:11.235274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:11.235274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:11.235274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:11.235274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:11.235274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:11.235274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:11.235274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:11.235274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:11.235274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:11.235274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:11.235274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:11.235274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:11.238617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 143/285] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:11.239629 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f1c73-d9b9-4780-93d3-87d81d608ca9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.278039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc16675-8801-4002-9aa7-c314fd62e3b3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.324775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f0a27-6cb8-432a-b381-c3d821e38a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.326163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0a27-6cb8-432a-b381-c3d821e38a1c None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.327447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0a27-6cb8-432a-b381-c3d821e38a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.327924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0a27-6cb8-432a-b381-c3d821e38a1c None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5/users/3a280523992140c9847411a56bb2b8f6/roles/34f334464e614ee784ee399c7a70719f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.330147 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f0a27-6cb8-432a-b381-c3d821e38a1c None admin] RBAC: Authorizing `identity:create_grant(project_id=73330eaa9cf649e283e878927d1e2da5, user_id=3a280523992140c9847411a56bb2b8f6, role_id=34f334464e614ee784ee399c7a70719f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.331819 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f0a27-6cb8-432a-b381-c3d821e38a1c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.348617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:11.350028 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 143/286] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:11 2022] PUT /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5/users/3a280523992140c9847411a56bb2b8f6/roles/34f334464e614ee784ee399c7a70719f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:11.351447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f0a27-6cb8-432a-b381-c3d821e38a1c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.366944 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f1c73-d9b9-4780-93d3-87d81d608ca9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.408388 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb289ab-fca3-4e6b-b1ea-0372bf392cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.409226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb289ab-fca3-4e6b-b1ea-0372bf392cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.409605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb289ab-fca3-4e6b-b1ea-0372bf392cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.410600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb289ab-fca3-4e6b-b1ea-0372bf392cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.411027 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb289ab-fca3-4e6b-b1ea-0372bf392cb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:11.411432 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb289ab-fca3-4e6b-b1ea-0372bf392cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.413439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb289ab-fca3-4e6b-b1ea-0372bf392cb6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.425458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 144/287] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:11.431196 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb289ab-fca3-4e6b-b1ea-0372bf392cb6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.440999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:11.490787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f0a27-6cb8-432a-b381-c3d821e38a1c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.529085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236fb76-871d-40c0-879c-7373231a6a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.529832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236fb76-871d-40c0-879c-7373231a6a8a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.530112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236fb76-871d-40c0-879c-7373231a6a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.530386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236fb76-871d-40c0-879c-7373231a6a8a None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.531195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236fb76-871d-40c0-879c-7373231a6a8a None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:11.532098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236fb76-871d-40c0-879c-7373231a6a8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.533507 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236fb76-871d-40c0-879c-7373231a6a8a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.543859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 144/288] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:11.550624 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236fb76-871d-40c0-879c-7373231a6a8a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.575275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb289ab-fca3-4e6b-b1ea-0372bf392cb6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.615968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec70ec-8fe8-4931-9bc6-6f914a62ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.616862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec70ec-8fe8-4931-9bc6-6f914a62ca52 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.617355 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec70ec-8fe8-4931-9bc6-6f914a62ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.617575 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec70ec-8fe8-4931-9bc6-6f914a62ca52 None admin] PATH_INFO: `/v3/users/5e07d418d127433fb229e108142d4434` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.634482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec70ec-8fe8-4931-9bc6-6f914a62ca52 None admin] RBAC: Authorizing `identity:get_user(user_id=5e07d418d127433fb229e108142d4434)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.636016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec70ec-8fe8-4931-9bc6-6f914a62ca52 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.638156 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 145/289] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:11 2022] GET /identity/v3/users/5e07d418d127433fb229e108142d4434 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:11.646291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec70ec-8fe8-4931-9bc6-6f914a62ca52 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.659146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:11.707306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236fb76-871d-40c0-879c-7373231a6a8a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.728759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:11.745029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545862cb-b287-482c-94e2-840f31ae85b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.745923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545862cb-b287-482c-94e2-840f31ae85b4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.746204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545862cb-b287-482c-94e2-840f31ae85b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.746711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545862cb-b287-482c-94e2-840f31ae85b4 None admin] PATH_INFO: `/v3/users/8ee4a92050584f8d9642d4310d7f7a86` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.747705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545862cb-b287-482c-94e2-840f31ae85b4 None admin] RBAC: Authorizing `identity:get_user(user_id=8ee4a92050584f8d9642d4310d7f7a86)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.749156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545862cb-b287-482c-94e2-840f31ae85b4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.751985 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 145/290] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:11 2022] GET /identity/v3/users/8ee4a92050584f8d9642d4310d7f7a86 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:11.759760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545862cb-b287-482c-94e2-840f31ae85b4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.762987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec70ec-8fe8-4931-9bc6-6f914a62ca52 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.800130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1983bc5-e678-41df-a9f3-3d3414fb0051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.800900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1983bc5-e678-41df-a9f3-3d3414fb0051 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.801296 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1983bc5-e678-41df-a9f3-3d3414fb0051 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.801578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1983bc5-e678-41df-a9f3-3d3414fb0051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.802407 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1983bc5-e678-41df-a9f3-3d3414fb0051 None admin] RBAC: Adding query filter params (role.id=7d15e336f8034a1fbdef48cfb7c4f717, user.id=5e07d418d127433fb229e108142d4434, scope.system=all) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:11.802714 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1983bc5-e678-41df-a9f3-3d3414fb0051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.804158 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1983bc5-e678-41df-a9f3-3d3414fb0051 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.809788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 146/291] 10.222.0.65 () {56 vars in 1308 bytes} [Thu Jun 23 10:49:11 2022] GET /identity/v3/role_assignments?user.id=5e07d418d127433fb229e108142d4434&scope.system=all&role.id=7d15e336f8034a1fbdef48cfb7c4f717 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:11.880071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545862cb-b287-482c-94e2-840f31ae85b4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.930794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37788519-4978-44d4-81fa-2fc2fe288d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.931450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37788519-4978-44d4-81fa-2fc2fe288d9a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.931759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37788519-4978-44d4-81fa-2fc2fe288d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.932041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37788519-4978-44d4-81fa-2fc2fe288d9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:12.016869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:12.037665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37788519-4978-44d4-81fa-2fc2fe288d9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:12.039166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37788519-4978-44d4-81fa-2fc2fe288d9a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:12.171019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 146/292] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:12.177522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1983bc5-e678-41df-a9f3-3d3414fb0051 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:12.271366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:12.304808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1983bc5-e678-41df-a9f3-3d3414fb0051 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:12.339049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad9138-393b-4a68-b460-86af504f8334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:12.339951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad9138-393b-4a68-b460-86af504f8334 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:12.340230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad9138-393b-4a68-b460-86af504f8334 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:12.340510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad9138-393b-4a68-b460-86af504f8334 None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:12.341681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad9138-393b-4a68-b460-86af504f8334 None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:12.343112 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad9138-393b-4a68-b460-86af504f8334 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:12.345158 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 147/293] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:12 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:12.350841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37788519-4978-44d4-81fa-2fc2fe288d9a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:12.504866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:12.518490 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37788519-4978-44d4-81fa-2fc2fe288d9a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:12.566577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a390fb-0d8c-408c-b90c-43472447e9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:12.567247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a390fb-0d8c-408c-b90c-43472447e9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:12.567532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a390fb-0d8c-408c-b90c-43472447e9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:12.567839 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a390fb-0d8c-408c-b90c-43472447e9b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:12.568741 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a390fb-0d8c-408c-b90c-43472447e9b5 None admin] RBAC: Adding query filter params (role.id=7d15e336f8034a1fbdef48cfb7c4f717, scope.project.id=73330eaa9cf649e283e878927d1e2da5, user.id=8ee4a92050584f8d9642d4310d7f7a86) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:12.569061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a390fb-0d8c-408c-b90c-43472447e9b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:12.570570 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a390fb-0d8c-408c-b90c-43472447e9b5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:12.580858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 147/294] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:12 2022] GET /identity/v3/role_assignments?user.id=8ee4a92050584f8d9642d4310d7f7a86&scope.project.id=73330eaa9cf649e283e878927d1e2da5&role.id=7d15e336f8034a1fbdef48cfb7c4f717 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:13.266480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f27e4-63f7-4673-8b96-3486ee8b371a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.266979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f27e4-63f7-4673-8b96-3486ee8b371a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.267085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f27e4-63f7-4673-8b96-3486ee8b371a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.268621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 148/295] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:13.273721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31398d-4b44-4039-b41b-04020f59f508 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.274015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31398d-4b44-4039-b41b-04020f59f508 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.274291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31398d-4b44-4039-b41b-04020f59f508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.292018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1e31398d-4b44-4039-b41b-04020f59f508 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:13.293215 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1e31398d-4b44-4039-b41b-04020f59f508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:13.360890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a0c28-2ead-4f06-a849-b6a8b057f566 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.361171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a0c28-2ead-4f06-a849-b6a8b057f566 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.361364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a0c28-2ead-4f06-a849-b6a8b057f566 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.362868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 149/296] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:13.365604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 148/297] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:13.367610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf71b3b-5d50-41d3-82ea-e9f971bbabc7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.367930 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf71b3b-5d50-41d3-82ea-e9f971bbabc7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.368211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf71b3b-5d50-41d3-82ea-e9f971bbabc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.384518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-adf71b3b-5d50-41d3-82ea-e9f971bbabc7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:13.385700 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-adf71b3b-5d50-41d3-82ea-e9f971bbabc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:13.390051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31398d-4b44-4039-b41b-04020f59f508 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:13.465442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 150/298] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:13.480658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:13.491323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf71b3b-5d50-41d3-82ea-e9f971bbabc7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:13.503376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31398d-4b44-4039-b41b-04020f59f508 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:13.541508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd8e70-df28-4053-8b27-cd51df2dc0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:13.542699 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd8e70-df28-4053-8b27-cd51df2dc0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.543221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd8e70-df28-4053-8b27-cd51df2dc0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.543718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd8e70-df28-4053-8b27-cd51df2dc0c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.555375 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd8e70-df28-4053-8b27-cd51df2dc0c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3bbd8e70-df28-4053-8b27-cd51df2dc0c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:13.563277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:13.564209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:13.565016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:13.565016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:13.565016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:13.565016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:13.565016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:13.565016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:13.565016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:13.565016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:13.565016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:13.565016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:13.565016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:13.565016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:13.567721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 149/299] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:13.573234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd8e70-df28-4053-8b27-cd51df2dc0c9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:13.581201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:13.626357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf71b3b-5d50-41d3-82ea-e9f971bbabc7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:13.677722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a7440-1577-4038-b207-8224a51839e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:13.678523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a7440-1577-4038-b207-8224a51839e3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.678805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a7440-1577-4038-b207-8224a51839e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.679082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a7440-1577-4038-b207-8224a51839e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.680404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a7440-1577-4038-b207-8224a51839e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:13.681842 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a7440-1577-4038-b207-8224a51839e3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:13.683731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 151/300] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:13.688790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a7440-1577-4038-b207-8224a51839e3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:13.695470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd8e70-df28-4053-8b27-cd51df2dc0c9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:13.742377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee361e-81dc-406f-9475-7e3ba3ad6255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:13.743412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee361e-81dc-406f-9475-7e3ba3ad6255 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.744340 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee361e-81dc-406f-9475-7e3ba3ad6255 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.744817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee361e-81dc-406f-9475-7e3ba3ad6255 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.746041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee361e-81dc-406f-9475-7e3ba3ad6255 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:13.746701 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee361e-81dc-406f-9475-7e3ba3ad6255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:13.748948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee361e-81dc-406f-9475-7e3ba3ad6255 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:13.761084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 150/301] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:13.766607 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee361e-81dc-406f-9475-7e3ba3ad6255 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:13.771408 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:13.806733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a7440-1577-4038-b207-8224a51839e3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:13.852580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac324a-f6ea-4952-9ea4-046acf6c9704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:13.853302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac324a-f6ea-4952-9ea4-046acf6c9704 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.853590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac324a-f6ea-4952-9ea4-046acf6c9704 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.853863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac324a-f6ea-4952-9ea4-046acf6c9704 None admin] PATH_INFO: `/v3/groups` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.854560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac324a-f6ea-4952-9ea4-046acf6c9704 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:13.856045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac324a-f6ea-4952-9ea4-046acf6c9704 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:13.866967 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:13.873550 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 152/302] 10.222.0.65 () {60 vars in 1134 bytes} [Thu Jun 23 10:49:13 2022] POST /identity/v3/groups => generated 229 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:49:13.875038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e17559-ca68-465b-b9b5-55b1ba68a1e5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.876099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e17559-ca68-465b-b9b5-55b1ba68a1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.876520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e17559-ca68-465b-b9b5-55b1ba68a1e5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.883093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 153/303] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:13.885296 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc9536-c02a-41b5-804b-41dafb59af81 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.885593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc9536-c02a-41b5-804b-41dafb59af81 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.885957 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc9536-c02a-41b5-804b-41dafb59af81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.897899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee361e-81dc-406f-9475-7e3ba3ad6255 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:13.911364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fedc9536-c02a-41b5-804b-41dafb59af81 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:13.912881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fedc9536-c02a-41b5-804b-41dafb59af81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:13.938424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c5d4b-f3f9-4285-891b-a781b09af0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:13.939373 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c5d4b-f3f9-4285-891b-a781b09af0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.939683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c5d4b-f3f9-4285-891b-a781b09af0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.940020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c5d4b-f3f9-4285-891b-a781b09af0f8 None admin] PATH_INFO: `/v3/users/af78b08ca68f4a4a841d212093d908dc` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.957066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c5d4b-f3f9-4285-891b-a781b09af0f8 None admin] RBAC: Authorizing `identity:get_user(user_id=af78b08ca68f4a4a841d212093d908dc)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:13.959416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c5d4b-f3f9-4285-891b-a781b09af0f8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:13.961001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 151/304] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:13 2022] GET /identity/v3/users/af78b08ca68f4a4a841d212093d908dc => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:13.966799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c5d4b-f3f9-4285-891b-a781b09af0f8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.004088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 154/305] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:14.022030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc9536-c02a-41b5-804b-41dafb59af81 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.042641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.103511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c5d4b-f3f9-4285-891b-a781b09af0f8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.150931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec176997-6d9a-4cca-9021-e1fb4764fde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.151847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec176997-6d9a-4cca-9021-e1fb4764fde6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.152277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec176997-6d9a-4cca-9021-e1fb4764fde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.152721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec176997-6d9a-4cca-9021-e1fb4764fde6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.152923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc9536-c02a-41b5-804b-41dafb59af81 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.153895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec176997-6d9a-4cca-9021-e1fb4764fde6 None admin] RBAC: Adding query filter params (role.id=34f334464e614ee784ee399c7a70719f, user.id=af78b08ca68f4a4a841d212093d908dc, scope.system=all) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:14.154375 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec176997-6d9a-4cca-9021-e1fb4764fde6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.156463 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec176997-6d9a-4cca-9021-e1fb4764fde6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.162055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 152/306] 10.222.0.65 () {56 vars in 1308 bytes} [Thu Jun 23 10:49:13 2022] GET /identity/v3/role_assignments?user.id=af78b08ca68f4a4a841d212093d908dc&scope.system=all&role.id=34f334464e614ee784ee399c7a70719f => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:14.166252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d39df-1dd7-42cd-98e8-afc1d4e877dd None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.166252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d39df-1dd7-42cd-98e8-afc1d4e877dd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.166252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d39df-1dd7-42cd-98e8-afc1d4e877dd None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.166252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 153/307] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:14.171129 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fedb3-0485-4b65-957a-99160303093b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.171602 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fedb3-0485-4b65-957a-99160303093b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.171975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fedb3-0485-4b65-957a-99160303093b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.186412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.195370 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c38fedb3-0485-4b65-957a-99160303093b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:14.196902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c38fedb3-0485-4b65-957a-99160303093b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:14.197908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedb19b-9b4e-4431-90a2-4bf6e00dae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.198785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedb19b-9b4e-4431-90a2-4bf6e00dae80 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.199070 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedb19b-9b4e-4431-90a2-4bf6e00dae80 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.199344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedb19b-9b4e-4431-90a2-4bf6e00dae80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.212178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedb19b-9b4e-4431-90a2-4bf6e00dae80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5fedb19b-9b4e-4431-90a2-4bf6e00dae80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:14.220304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:14.221248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:14.222270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 155/308] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:14.233768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedb19b-9b4e-4431-90a2-4bf6e00dae80 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.299589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 154/309] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:14.323029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fedb3-0485-4b65-957a-99160303093b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.362425 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedb19b-9b4e-4431-90a2-4bf6e00dae80 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.386276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.399114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32699ae8-3869-40d2-93d3-51f73c7a8c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.400159 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32699ae8-3869-40d2-93d3-51f73c7a8c8e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.400658 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32699ae8-3869-40d2-93d3-51f73c7a8c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.401193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32699ae8-3869-40d2-93d3-51f73c7a8c8e None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.402137 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32699ae8-3869-40d2-93d3-51f73c7a8c8e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:14.402635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32699ae8-3869-40d2-93d3-51f73c7a8c8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.404365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32699ae8-3869-40d2-93d3-51f73c7a8c8e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.413705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 156/310] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:14.418776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32699ae8-3869-40d2-93d3-51f73c7a8c8e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.440843 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fedb3-0485-4b65-957a-99160303093b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.483146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.485351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb076321-dd2b-49b2-9f46-9da5afdefec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.486255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb076321-dd2b-49b2-9f46-9da5afdefec3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.486604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb076321-dd2b-49b2-9f46-9da5afdefec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.486931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb076321-dd2b-49b2-9f46-9da5afdefec3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.502333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb076321-dd2b-49b2-9f46-9da5afdefec3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-fb076321-dd2b-49b2-9f46-9da5afdefec3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:14.512545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:14.513468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:14.514257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:14.514257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:14.514257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:14.514257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:14.514257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:14.514257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:14.514257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:14.514257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:14.514257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:14.514257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:14.514257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:14.514257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:14.519157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 155/311] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:14.520519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb076321-dd2b-49b2-9f46-9da5afdefec3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.550006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32699ae8-3869-40d2-93d3-51f73c7a8c8e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.594304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194ea72-37be-4746-a368-bdcf8d93a644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.595846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194ea72-37be-4746-a368-bdcf8d93a644 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.596235 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194ea72-37be-4746-a368-bdcf8d93a644 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.596610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194ea72-37be-4746-a368-bdcf8d93a644 None admin] PATH_INFO: `/v3/users/3a280523992140c9847411a56bb2b8f6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.598206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194ea72-37be-4746-a368-bdcf8d93a644 None admin] RBAC: Authorizing `identity:get_user(user_id=3a280523992140c9847411a56bb2b8f6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.599951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194ea72-37be-4746-a368-bdcf8d93a644 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.606543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 157/312] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/users/3a280523992140c9847411a56bb2b8f6 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:14.608140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194ea72-37be-4746-a368-bdcf8d93a644 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.644408 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb076321-dd2b-49b2-9f46-9da5afdefec3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.661953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.681863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67526c-13f5-4925-9798-48416c70cd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.682538 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67526c-13f5-4925-9798-48416c70cd90 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.682821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67526c-13f5-4925-9798-48416c70cd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.683099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67526c-13f5-4925-9798-48416c70cd90 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.683984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67526c-13f5-4925-9798-48416c70cd90 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:14.684298 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67526c-13f5-4925-9798-48416c70cd90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.685825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67526c-13f5-4925-9798-48416c70cd90 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.695464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 156/313] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:14.700627 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67526c-13f5-4925-9798-48416c70cd90 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.738775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194ea72-37be-4746-a368-bdcf8d93a644 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.756124 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.783594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9eb41-0dcb-4b47-9e79-96d375f8fa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.785251 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9eb41-0dcb-4b47-9e79-96d375f8fa7a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.785764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9eb41-0dcb-4b47-9e79-96d375f8fa7a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.786510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9eb41-0dcb-4b47-9e79-96d375f8fa7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.820446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67526c-13f5-4925-9798-48416c70cd90 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.855807 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326211ba-987a-4785-8090-bdb48c83e357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.857105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326211ba-987a-4785-8090-bdb48c83e357 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.857215 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326211ba-987a-4785-8090-bdb48c83e357 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.857426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326211ba-987a-4785-8090-bdb48c83e357 None admin] PATH_INFO: `/v3/users/5e07d418d127433fb229e108142d4434` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.858938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326211ba-987a-4785-8090-bdb48c83e357 None admin] RBAC: Authorizing `identity:get_user(user_id=5e07d418d127433fb229e108142d4434)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.861084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326211ba-987a-4785-8090-bdb48c83e357 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.862885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 157/314] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/users/5e07d418d127433fb229e108142d4434 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:14.868551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326211ba-987a-4785-8090-bdb48c83e357 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.876718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9eb41-0dcb-4b47-9e79-96d375f8fa7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.878427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9eb41-0dcb-4b47-9e79-96d375f8fa7a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.925084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.983585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326211ba-987a-4785-8090-bdb48c83e357 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:15.019005 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1f065-23e2-4a6f-8b27-c15af96be0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:15.019910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1f065-23e2-4a6f-8b27-c15af96be0c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:15.020197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1f065-23e2-4a6f-8b27-c15af96be0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:15.020482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1f065-23e2-4a6f-8b27-c15af96be0c9 None admin] PATH_INFO: `/v3/system/users/5e07d418d127433fb229e108142d4434/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:15.021532 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 158/315] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:15.027708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1f065-23e2-4a6f-8b27-c15af96be0c9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5e07d418d127433fb229e108142d4434, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:15.027708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1f065-23e2-4a6f-8b27-c15af96be0c9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:15.027907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9eb41-0dcb-4b47-9e79-96d375f8fa7a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:15.037715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 158/316] 10.222.0.65 () {58 vars in 1334 bytes} [Thu Jun 23 10:49:14 2022] PUT /identity/v3/system/users/5e07d418d127433fb229e108142d4434/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:15.059828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:15.160584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9eb41-0dcb-4b47-9e79-96d375f8fa7a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:15.199458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd5cf2-a480-4997-8c1e-f4e468fac299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:15.200265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd5cf2-a480-4997-8c1e-f4e468fac299 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:15.200546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd5cf2-a480-4997-8c1e-f4e468fac299 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:15.200819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd5cf2-a480-4997-8c1e-f4e468fac299 None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:15.201982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd5cf2-a480-4997-8c1e-f4e468fac299 None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:15.203984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd5cf2-a480-4997-8c1e-f4e468fac299 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:15.206036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 159/317] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:15 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:15.211739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1f065-23e2-4a6f-8b27-c15af96be0c9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:15.316096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1f065-23e2-4a6f-8b27-c15af96be0c9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:15.327424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:15.357515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da8961-d2f0-490d-9707-2b1ec850e209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:15.358767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da8961-d2f0-490d-9707-2b1ec850e209 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:15.359299 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da8961-d2f0-490d-9707-2b1ec850e209 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:15.359802 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da8961-d2f0-490d-9707-2b1ec850e209 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:15.360858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da8961-d2f0-490d-9707-2b1ec850e209 None admin] RBAC: Adding query filter params (role.id=34f334464e614ee784ee399c7a70719f, scope.project.id=73330eaa9cf649e283e878927d1e2da5, user.id=3a280523992140c9847411a56bb2b8f6) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:15.361369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da8961-d2f0-490d-9707-2b1ec850e209 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:15.363009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da8961-d2f0-490d-9707-2b1ec850e209 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:15.369709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 159/318] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:15 2022] GET /identity/v3/role_assignments?user.id=3a280523992140c9847411a56bb2b8f6&scope.project.id=73330eaa9cf649e283e878927d1e2da5&role.id=34f334464e614ee784ee399c7a70719f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:16.011790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efec6e-5157-4777-a87e-02cd0b6feebd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.012331 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efec6e-5157-4777-a87e-02cd0b6feebd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.012331 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efec6e-5157-4777-a87e-02cd0b6feebd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.013854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 160/319] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:16.018846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb489ff-f7cd-45de-80a0-3680b0f1513c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.019229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb489ff-f7cd-45de-80a0-3680b0f1513c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.019393 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb489ff-f7cd-45de-80a0-3680b0f1513c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.035977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9fb489ff-f7cd-45de-80a0-3680b0f1513c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:16.037169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9fb489ff-f7cd-45de-80a0-3680b0f1513c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:16.112091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 160/320] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:16.133873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efec6e-5157-4777-a87e-02cd0b6feebd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:16.225600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:16.238856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cbfdd-9c16-4f19-b9f9-c4e48c3f1bf6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.239149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cbfdd-9c16-4f19-b9f9-c4e48c3f1bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.239439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cbfdd-9c16-4f19-b9f9-c4e48c3f1bf6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.240839 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 161/321] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:16.244384 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efec6e-5157-4777-a87e-02cd0b6feebd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:16.245673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013528be-c933-4e9d-8050-ddfc9530c3d5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.245955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013528be-c933-4e9d-8050-ddfc9530c3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.246228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013528be-c933-4e9d-8050-ddfc9530c3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.262949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-013528be-c933-4e9d-8050-ddfc9530c3d5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:16.264081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-013528be-c933-4e9d-8050-ddfc9530c3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:16.282854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915d5b5-dced-411f-b092-0b7895cf9abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:16.284073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915d5b5-dced-411f-b092-0b7895cf9abd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.284559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915d5b5-dced-411f-b092-0b7895cf9abd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.285042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915d5b5-dced-411f-b092-0b7895cf9abd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.286282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915d5b5-dced-411f-b092-0b7895cf9abd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:16.287746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915d5b5-dced-411f-b092-0b7895cf9abd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:16.289377 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 161/322] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:16.294713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915d5b5-dced-411f-b092-0b7895cf9abd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:16.337898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 162/323] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:16.360188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013528be-c933-4e9d-8050-ddfc9530c3d5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:16.394261 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:16.400774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915d5b5-dced-411f-b092-0b7895cf9abd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:16.435735 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b2717-c82f-4c7c-bff8-55ff01bb1652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:16.436400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b2717-c82f-4c7c-bff8-55ff01bb1652 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.436685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b2717-c82f-4c7c-bff8-55ff01bb1652 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.436961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b2717-c82f-4c7c-bff8-55ff01bb1652 None admin] PATH_INFO: `/v3/groups` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.438192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b2717-c82f-4c7c-bff8-55ff01bb1652 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:16.439622 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b2717-c82f-4c7c-bff8-55ff01bb1652 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:16.455028 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 162/324] 10.222.0.65 () {60 vars in 1134 bytes} [Thu Jun 23 10:49:16 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:49:16.474663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013528be-c933-4e9d-8050-ddfc9530c3d5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:16.514667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f31b8-4f95-4554-a191-d043687510d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:16.516457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f31b8-4f95-4554-a191-d043687510d4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.516938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f31b8-4f95-4554-a191-d043687510d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.517468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f31b8-4f95-4554-a191-d043687510d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.529475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f31b8-4f95-4554-a191-d043687510d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-613f31b8-4f95-4554-a191-d043687510d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:16.538013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:16.538958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:16.539779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:16.539779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:16.539779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:16.539779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:16.539779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:16.539779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:16.539779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:16.539779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:16.539779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:16.539779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:16.539779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:16.539779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:16.546012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 163/325] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:16.547783 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b2717-c82f-4c7c-bff8-55ff01bb1652 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:16.565792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:16.660197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b2717-c82f-4c7c-bff8-55ff01bb1652 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:16.696565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d40142-5778-498f-8b4f-43a858894aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:16.697247 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d40142-5778-498f-8b4f-43a858894aed None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.697533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d40142-5778-498f-8b4f-43a858894aed None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.697816 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d40142-5778-498f-8b4f-43a858894aed None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.698584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d40142-5778-498f-8b4f-43a858894aed None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:16.698894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d40142-5778-498f-8b4f-43a858894aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:16.700586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d40142-5778-498f-8b4f-43a858894aed None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:16.710466 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 163/326] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:16.716714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f31b8-4f95-4554-a191-d043687510d4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:16.818222 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:16.826476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f31b8-4f95-4554-a191-d043687510d4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:16.865051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217466e8-231f-4782-b958-0bf597fab801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:16.866305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217466e8-231f-4782-b958-0bf597fab801 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.866760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217466e8-231f-4782-b958-0bf597fab801 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.867229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217466e8-231f-4782-b958-0bf597fab801 None admin] PATH_INFO: `/v3/users/af78b08ca68f4a4a841d212093d908dc` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.868675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217466e8-231f-4782-b958-0bf597fab801 None admin] RBAC: Authorizing `identity:get_user(user_id=af78b08ca68f4a4a841d212093d908dc)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:16.870407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217466e8-231f-4782-b958-0bf597fab801 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:16.872641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 164/327] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:16 2022] GET /identity/v3/users/af78b08ca68f4a4a841d212093d908dc => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:16.879832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d40142-5778-498f-8b4f-43a858894aed None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:16.969735 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:16.996276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d40142-5778-498f-8b4f-43a858894aed None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:17.036873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531660e4-5eb6-4387-818c-06eeac08686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:17.038452 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531660e4-5eb6-4387-818c-06eeac08686c None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.038886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531660e4-5eb6-4387-818c-06eeac08686c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.039340 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531660e4-5eb6-4387-818c-06eeac08686c None admin] PATH_INFO: `/v3/system/users/af78b08ca68f4a4a841d212093d908dc/roles/34f334464e614ee784ee399c7a70719f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.041072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531660e4-5eb6-4387-818c-06eeac08686c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=af78b08ca68f4a4a841d212093d908dc, role_id=34f334464e614ee784ee399c7a70719f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:17.042615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531660e4-5eb6-4387-818c-06eeac08686c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:17.054891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 164/328] 10.222.0.65 () {58 vars in 1334 bytes} [Thu Jun 23 10:49:16 2022] PUT /identity/v3/system/users/af78b08ca68f4a4a841d212093d908dc/roles/34f334464e614ee784ee399c7a70719f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:49:17.149459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a941e-dec5-4381-813c-e7d39c9d0be1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.150014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a941e-dec5-4381-813c-e7d39c9d0be1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.150014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a941e-dec5-4381-813c-e7d39c9d0be1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.151682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 165/329] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:17.158004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732875b3-4f4f-4bd2-af71-181df07d4f77 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.158315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732875b3-4f4f-4bd2-af71-181df07d4f77 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.158579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732875b3-4f4f-4bd2-af71-181df07d4f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.182005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-732875b3-4f4f-4bd2-af71-181df07d4f77 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:17.183298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-732875b3-4f4f-4bd2-af71-181df07d4f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:17.271796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 165/330] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:17.296168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a941e-dec5-4381-813c-e7d39c9d0be1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:17.428492 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a941e-dec5-4381-813c-e7d39c9d0be1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:17.469136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4884f-3c25-4ade-9494-b45e125ecfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:17.469983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4884f-3c25-4ade-9494-b45e125ecfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.470276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4884f-3c25-4ade-9494-b45e125ecfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.470551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4884f-3c25-4ade-9494-b45e125ecfc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.484652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4884f-3c25-4ade-9494-b45e125ecfc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6ca4884f-3c25-4ade-9494-b45e125ecfc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:17.492530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:17.493407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:17.494177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 166/331] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:17.501886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732875b3-4f4f-4bd2-af71-181df07d4f77 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:17.591800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:17.634095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732875b3-4f4f-4bd2-af71-181df07d4f77 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:17.675936 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1aaa1-8eed-4990-94bf-e4b12c8825d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:17.676611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1aaa1-8eed-4990-94bf-e4b12c8825d3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.676890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1aaa1-8eed-4990-94bf-e4b12c8825d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.677160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1aaa1-8eed-4990-94bf-e4b12c8825d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.677919 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1aaa1-8eed-4990-94bf-e4b12c8825d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:17.678223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1aaa1-8eed-4990-94bf-e4b12c8825d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:17.679704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1aaa1-8eed-4990-94bf-e4b12c8825d3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:17.692097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 166/332] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:17.697610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4884f-3c25-4ade-9494-b45e125ecfc9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:17.746430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:17.865959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4884f-3c25-4ade-9494-b45e125ecfc9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:17.901765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29b729-b5ec-422a-addf-7722c1a18ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:17.903044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29b729-b5ec-422a-addf-7722c1a18ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.903553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29b729-b5ec-422a-addf-7722c1a18ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.904054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29b729-b5ec-422a-addf-7722c1a18ab4 None admin] PATH_INFO: `/v3/users/5e07d418d127433fb229e108142d4434` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.908522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29b729-b5ec-422a-addf-7722c1a18ab4 None admin] RBAC: Authorizing `identity:get_user(user_id=5e07d418d127433fb229e108142d4434)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:17.910483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29b729-b5ec-422a-addf-7722c1a18ab4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:17.912523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 167/333] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:17 2022] GET /identity/v3/users/5e07d418d127433fb229e108142d4434 => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:17.918520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1aaa1-8eed-4990-94bf-e4b12c8825d3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:18.057087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1aaa1-8eed-4990-94bf-e4b12c8825d3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:18.087027 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:18.101077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1781ca2-bea7-4f2b-95bf-ba99589af218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:18.101756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1781ca2-bea7-4f2b-95bf-ba99589af218 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:18.102042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1781ca2-bea7-4f2b-95bf-ba99589af218 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:18.102321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1781ca2-bea7-4f2b-95bf-ba99589af218 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:18.103171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1781ca2-bea7-4f2b-95bf-ba99589af218 None admin] RBAC: Adding query filter params (role.id=7d15e336f8034a1fbdef48cfb7c4f717, user.id=5e07d418d127433fb229e108142d4434, scope.system=all) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:18.103488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1781ca2-bea7-4f2b-95bf-ba99589af218 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:18.104984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1781ca2-bea7-4f2b-95bf-ba99589af218 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:18.127404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 167/334] 10.222.0.65 () {56 vars in 1308 bytes} [Thu Jun 23 10:49:17 2022] GET /identity/v3/role_assignments?user.id=5e07d418d127433fb229e108142d4434&scope.system=all&role.id=7d15e336f8034a1fbdef48cfb7c4f717 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:19.357576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ad86d-e21f-4b55-9bb5-ef0cbd8355da None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.358056 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ad86d-e21f-4b55-9bb5-ef0cbd8355da None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.358056 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ad86d-e21f-4b55-9bb5-ef0cbd8355da None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.359993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 168/335] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:19.377004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2f51c-437b-4409-9ef2-cda58ae36eeb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.377911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2f51c-437b-4409-9ef2-cda58ae36eeb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.378384 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2f51c-437b-4409-9ef2-cda58ae36eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.404664 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5ca2f51c-437b-4409-9ef2-cda58ae36eeb None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:19.406249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5ca2f51c-437b-4409-9ef2-cda58ae36eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:19.490357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 168/336] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:19.537936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ad86d-e21f-4b55-9bb5-ef0cbd8355da None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:19.617346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0cd40-6d25-47ac-9b84-62b71cefa7ea None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.617632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0cd40-6d25-47ac-9b84-62b71cefa7ea None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.617820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0cd40-6d25-47ac-9b84-62b71cefa7ea None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.619422 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 169/337] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:19.624423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e60ec6-1f57-4e13-8120-48af1132468d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.625274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e60ec6-1f57-4e13-8120-48af1132468d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.625558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e60ec6-1f57-4e13-8120-48af1132468d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.644040 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-13e60ec6-1f57-4e13-8120-48af1132468d None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:19.645243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-13e60ec6-1f57-4e13-8120-48af1132468d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:19.673043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:19.694050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ad86d-e21f-4b55-9bb5-ef0cbd8355da None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:19.768594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b661bd-4dda-4a21-8a7e-3293b7b4ea9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:19.769415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b661bd-4dda-4a21-8a7e-3293b7b4ea9b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.769701 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b661bd-4dda-4a21-8a7e-3293b7b4ea9b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.769982 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b661bd-4dda-4a21-8a7e-3293b7b4ea9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.786133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 170/338] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:19.790250 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b661bd-4dda-4a21-8a7e-3293b7b4ea9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-43b661bd-4dda-4a21-8a7e-3293b7b4ea9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:19.803106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:19.804071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:19.805284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 169/339] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:19.811833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e60ec6-1f57-4e13-8120-48af1132468d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:19.873929 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:19.905221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b661bd-4dda-4a21-8a7e-3293b7b4ea9b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:19.937975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e60ec6-1f57-4e13-8120-48af1132468d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:19.984877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccf4b7-fc4c-44ac-8586-5f8b03a50dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:19.985557 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccf4b7-fc4c-44ac-8586-5f8b03a50dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.985837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccf4b7-fc4c-44ac-8586-5f8b03a50dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.986112 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccf4b7-fc4c-44ac-8586-5f8b03a50dc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.990793 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccf4b7-fc4c-44ac-8586-5f8b03a50dc8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:19.991113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccf4b7-fc4c-44ac-8586-5f8b03a50dc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:19.997169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccf4b7-fc4c-44ac-8586-5f8b03a50dc8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:20.008410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 171/340] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:20.009895 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625dd85-0fa3-4231-85fe-60705bb90222 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.010186 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625dd85-0fa3-4231-85fe-60705bb90222 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.010579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625dd85-0fa3-4231-85fe-60705bb90222 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.012085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 172/341] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:20.021127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3b757-de28-43d5-a0d4-0097d3fb86d7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.021420 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3b757-de28-43d5-a0d4-0097d3fb86d7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.021705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3b757-de28-43d5-a0d4-0097d3fb86d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.042740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-41c3b757-de28-43d5-a0d4-0097d3fb86d7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:20.048093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-41c3b757-de28-43d5-a0d4-0097d3fb86d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:20.085824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b661bd-4dda-4a21-8a7e-3293b7b4ea9b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:20.134955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a2483-cfbc-4b7f-b3ae-1478ceb71909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:20.135737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a2483-cfbc-4b7f-b3ae-1478ceb71909 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.136027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a2483-cfbc-4b7f-b3ae-1478ceb71909 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.136296 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a2483-cfbc-4b7f-b3ae-1478ceb71909 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.154353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 173/342] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:20.155282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3b757-de28-43d5-a0d4-0097d3fb86d7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:20.168318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a2483-cfbc-4b7f-b3ae-1478ceb71909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1b8a2483-cfbc-4b7f-b3ae-1478ceb71909 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:20.183682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:20.184605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:20.185382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 170/343] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:20.186016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a2483-cfbc-4b7f-b3ae-1478ceb71909 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:20.206964 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:20.240023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:20.325792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3b757-de28-43d5-a0d4-0097d3fb86d7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:20.369317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2046d6a-7b91-4e63-a512-f300c7a11f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:20.370228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2046d6a-7b91-4e63-a512-f300c7a11f8e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.370509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2046d6a-7b91-4e63-a512-f300c7a11f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.371200 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2046d6a-7b91-4e63-a512-f300c7a11f8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.373704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a2483-cfbc-4b7f-b3ae-1478ceb71909 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:20.433979 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabe536-ce4d-432e-9d3e-a1dbb36c41b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:20.435092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabe536-ce4d-432e-9d3e-a1dbb36c41b1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.435572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabe536-ce4d-432e-9d3e-a1dbb36c41b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.436389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabe536-ce4d-432e-9d3e-a1dbb36c41b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.465225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabe536-ce4d-432e-9d3e-a1dbb36c41b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:20.475810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2046d6a-7b91-4e63-a512-f300c7a11f8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:20.477349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2046d6a-7b91-4e63-a512-f300c7a11f8e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8dabe536-ce4d-432e-9d3e-a1dbb36c41b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:20.478372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:20.479494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:20.480612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:20.480612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:20.480612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:20.480612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:20.480612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:20.480612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:20.480612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:20.480612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:20.480612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:20.480612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:20.480612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:20.480612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:20.490086 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 171/344] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:20.491153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabe536-ce4d-432e-9d3e-a1dbb36c41b1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:20.542600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:20.597299 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:20.622337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 174/345] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:20.624435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b101978-b00c-436b-acac-ce3b5bd7a501 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.624780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b101978-b00c-436b-acac-ce3b5bd7a501 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.625060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b101978-b00c-436b-acac-ce3b5bd7a501 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.627386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 175/346] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:20.628285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b101978-b00c-436b-acac-ce3b5bd7a501 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:20.642070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabe536-ce4d-432e-9d3e-a1dbb36c41b1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:20.689131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168f889-1565-4570-9fe8-8b9adc06202c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:20.689131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168f889-1565-4570-9fe8-8b9adc06202c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.690495 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168f889-1565-4570-9fe8-8b9adc06202c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.690780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168f889-1565-4570-9fe8-8b9adc06202c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.691531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168f889-1565-4570-9fe8-8b9adc06202c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:20.691868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168f889-1565-4570-9fe8-8b9adc06202c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:20.693247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168f889-1565-4570-9fe8-8b9adc06202c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:20.703135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 172/347] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:20.703983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2d24c-2562-41fb-8278-a3daa540e2b7 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.704271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2d24c-2562-41fb-8278-a3daa540e2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.704543 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2d24c-2562-41fb-8278-a3daa540e2b7 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.709830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 173/348] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:20.710609 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2d24c-2562-41fb-8278-a3daa540e2b7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:20.783710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b101978-b00c-436b-acac-ce3b5bd7a501 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:20.828563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788473fe-be42-46e7-8b5c-ff19ba495e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:20.829338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788473fe-be42-46e7-8b5c-ff19ba495e80 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.829710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788473fe-be42-46e7-8b5c-ff19ba495e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.830078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788473fe-be42-46e7-8b5c-ff19ba495e80 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.830891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788473fe-be42-46e7-8b5c-ff19ba495e80 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:20.831201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788473fe-be42-46e7-8b5c-ff19ba495e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:20.832593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788473fe-be42-46e7-8b5c-ff19ba495e80 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:20.842431 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 176/349] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:20.843762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2de000-2f56-485a-9325-5036b9759f64 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.844055 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2de000-2f56-485a-9325-5036b9759f64 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.844329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2de000-2f56-485a-9325-5036b9759f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.847945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2d24c-2562-41fb-8278-a3daa540e2b7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:20.852894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:20.864168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4f2de000-2f56-485a-9325-5036b9759f64 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:20.865470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f2de000-2f56-485a-9325-5036b9759f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:20.884136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:20.890885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa92a3-6c1e-4bf4-b552-a9d18e4ce728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:20.892941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa92a3-6c1e-4bf4-b552-a9d18e4ce728 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.893411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa92a3-6c1e-4bf4-b552-a9d18e4ce728 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.893868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa92a3-6c1e-4bf4-b552-a9d18e4ce728 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.896439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa92a3-6c1e-4bf4-b552-a9d18e4ce728 None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:20.897524 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa92a3-6c1e-4bf4-b552-a9d18e4ce728 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:20.899923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 174/350] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:20 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:20.900533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa92a3-6c1e-4bf4-b552-a9d18e4ce728 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:20.954266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 177/351] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:20.954266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8a47a-1100-4f56-a241-daa566e76a78 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.954266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8a47a-1100-4f56-a241-daa566e76a78 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.954266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8a47a-1100-4f56-a241-daa566e76a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.976721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a2a8a47a-1100-4f56-a241-daa566e76a78 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:20.978179 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a2a8a47a-1100-4f56-a241-daa566e76a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:21.029766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa92a3-6c1e-4bf4-b552-a9d18e4ce728 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.070762 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773be05-24c2-463e-b77f-21ed0519f379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.071562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773be05-24c2-463e-b77f-21ed0519f379 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.072737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773be05-24c2-463e-b77f-21ed0519f379 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.072737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773be05-24c2-463e-b77f-21ed0519f379 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.087790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 178/352] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:21.088720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8a47a-1100-4f56-a241-daa566e76a78 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.154474 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:21.157829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773be05-24c2-463e-b77f-21ed0519f379 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:21.159831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:21.160538 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773be05-24c2-463e-b77f-21ed0519f379 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:21.215926 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8a47a-1100-4f56-a241-daa566e76a78 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.267102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4c653-16b8-4171-a547-9a719a3f5d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.268801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4c653-16b8-4171-a547-9a719a3f5d9e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.269254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4c653-16b8-4171-a547-9a719a3f5d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.269817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4c653-16b8-4171-a547-9a719a3f5d9e None admin] PATH_INFO: `/v3/users/af78b08ca68f4a4a841d212093d908dc` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.271829 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4c653-16b8-4171-a547-9a719a3f5d9e None admin] RBAC: Authorizing `identity:get_user(user_id=af78b08ca68f4a4a841d212093d908dc)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:21.273860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4c653-16b8-4171-a547-9a719a3f5d9e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:21.275952 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 179/353] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/users/af78b08ca68f4a4a841d212093d908dc => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:21.276869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4c653-16b8-4171-a547-9a719a3f5d9e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.281938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 175/354] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:21.282771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773be05-24c2-463e-b77f-21ed0519f379 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.403031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:21.407978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773be05-24c2-463e-b77f-21ed0519f379 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.411317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4c653-16b8-4171-a547-9a719a3f5d9e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.446889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b6cf2-bb25-4940-987e-a71d5b19bd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.447727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b6cf2-bb25-4940-987e-a71d5b19bd7d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.448014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b6cf2-bb25-4940-987e-a71d5b19bd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.448290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b6cf2-bb25-4940-987e-a71d5b19bd7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.450259 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:21.458278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe1290-35d6-438a-90a2-1010a7d2b756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.458937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b6cf2-bb25-4940-987e-a71d5b19bd7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:21.459284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe1290-35d6-438a-90a2-1010a7d2b756 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.459571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe1290-35d6-438a-90a2-1010a7d2b756 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.459882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe1290-35d6-438a-90a2-1010a7d2b756 None admin] PATH_INFO: `/v3/groups/5c4af3860a484ddda332a48e2765d284` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4b7b6cf2-bb25-4940-987e-a71d5b19bd7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:21.467160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:21.468266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:21.469091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 176/355] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:21.469761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b6cf2-bb25-4940-987e-a71d5b19bd7d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.471278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe1290-35d6-438a-90a2-1010a7d2b756 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4af3860a484ddda332a48e2765d284)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:21.473086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe1290-35d6-438a-90a2-1010a7d2b756 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:21.476911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 180/356] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/groups/5c4af3860a484ddda332a48e2765d284 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:21.476911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe1290-35d6-438a-90a2-1010a7d2b756 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.597335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b6cf2-bb25-4940-987e-a71d5b19bd7d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.602922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe1290-35d6-438a-90a2-1010a7d2b756 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.640178 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6339540-9e48-446e-af37-ec8c9bb967e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.640988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6339540-9e48-446e-af37-ec8c9bb967e9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.641274 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6339540-9e48-446e-af37-ec8c9bb967e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.641552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6339540-9e48-446e-af37-ec8c9bb967e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.642099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff8ce2-87ef-44d6-8b4e-1317b3a2aa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.642756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff8ce2-87ef-44d6-8b4e-1317b3a2aa22 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.643040 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff8ce2-87ef-44d6-8b4e-1317b3a2aa22 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.643323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff8ce2-87ef-44d6-8b4e-1317b3a2aa22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.644193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff8ce2-87ef-44d6-8b4e-1317b3a2aa22 None admin] RBAC: Adding query filter params (role.id=34f334464e614ee784ee399c7a70719f, user.id=af78b08ca68f4a4a841d212093d908dc, scope.system=all) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:21.644510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff8ce2-87ef-44d6-8b4e-1317b3a2aa22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:21.645940 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff8ce2-87ef-44d6-8b4e-1317b3a2aa22 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:21.655166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6339540-9e48-446e-af37-ec8c9bb967e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:21.659642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 181/357] 10.222.0.65 () {56 vars in 1308 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/role_assignments?user.id=af78b08ca68f4a4a841d212093d908dc&scope.system=all&role.id=34f334464e614ee784ee399c7a70719f => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:21.660558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff8ce2-87ef-44d6-8b4e-1317b3a2aa22 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e6339540-9e48-446e-af37-ec8c9bb967e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:21.664704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:21.665659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 177/358] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:21.672272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caad54-e259-4218-b093-3bc6c31388bd None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.673183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caad54-e259-4218-b093-3bc6c31388bd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.673183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caad54-e259-4218-b093-3bc6c31388bd None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.673183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 178/359] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:21.673183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caad54-e259-4218-b093-3bc6c31388bd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.705152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:21.741839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:21.805933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff8ce2-87ef-44d6-8b4e-1317b3a2aa22 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.818986 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caad54-e259-4218-b093-3bc6c31388bd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.853201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5937dae-5ae5-4131-b881-d5a4f6f51ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.854236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5937dae-5ae5-4131-b881-d5a4f6f51ffa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.854695 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5937dae-5ae5-4131-b881-d5a4f6f51ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.854820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5937dae-5ae5-4131-b881-d5a4f6f51ffa None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.856077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5937dae-5ae5-4131-b881-d5a4f6f51ffa None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:21.857643 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5937dae-5ae5-4131-b881-d5a4f6f51ffa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:21.860691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 182/360] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:21.861264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5937dae-5ae5-4131-b881-d5a4f6f51ffa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.861758 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bb0ec-c9b3-4e27-8998-63cf797fe6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.862439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bb0ec-c9b3-4e27-8998-63cf797fe6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.862725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bb0ec-c9b3-4e27-8998-63cf797fe6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.863003 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bb0ec-c9b3-4e27-8998-63cf797fe6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.863930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bb0ec-c9b3-4e27-8998-63cf797fe6a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:21.864121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bb0ec-c9b3-4e27-8998-63cf797fe6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:21.865932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bb0ec-c9b3-4e27-8998-63cf797fe6a9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:21.876964 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 179/361] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:21.879005 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef81da-628d-4a05-acf3-a1ac6f631c4e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.879482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef81da-628d-4a05-acf3-a1ac6f631c4e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.879785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef81da-628d-4a05-acf3-a1ac6f631c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.898313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3aef81da-628d-4a05-acf3-a1ac6f631c4e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:21.899630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3aef81da-628d-4a05-acf3-a1ac6f631c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:21.978179 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5937dae-5ae5-4131-b881-d5a4f6f51ffa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.978516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 180/362] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:21.979272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef81da-628d-4a05-acf3-a1ac6f631c4e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.001837 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:22.010448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:22.025889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c6508-e040-46ed-9b11-b5cebdd2d7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.026268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c6508-e040-46ed-9b11-b5cebdd2d7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.026560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c6508-e040-46ed-9b11-b5cebdd2d7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.026839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c6508-e040-46ed-9b11-b5cebdd2d7c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.027776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c6508-e040-46ed-9b11-b5cebdd2d7c2 None admin] RBAC: Adding query filter params (group.id=5c4af3860a484ddda332a48e2765d284, role.id=7d15e336f8034a1fbdef48cfb7c4f717, scope.project.id=2f8281ddbf1345088c2c4475cad30949) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:22.028034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c6508-e040-46ed-9b11-b5cebdd2d7c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:22.029457 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c6508-e040-46ed-9b11-b5cebdd2d7c2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:22.035721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 183/363] 10.222.0.65 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/role_assignments?group.id=5c4af3860a484ddda332a48e2765d284&scope.project.id=2f8281ddbf1345088c2c4475cad30949&role.id=7d15e336f8034a1fbdef48cfb7c4f717 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:22.036532 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c6508-e040-46ed-9b11-b5cebdd2d7c2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.122440 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef81da-628d-4a05-acf3-a1ac6f631c4e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.169752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde18540-c3a4-4462-b431-ad2eb21ebcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.170447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde18540-c3a4-4462-b431-ad2eb21ebcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.172586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde18540-c3a4-4462-b431-ad2eb21ebcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.172586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde18540-c3a4-4462-b431-ad2eb21ebcf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.172586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde18540-c3a4-4462-b431-ad2eb21ebcf0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:22.172808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde18540-c3a4-4462-b431-ad2eb21ebcf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:22.175698 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde18540-c3a4-4462-b431-ad2eb21ebcf0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:22.188424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 181/364] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:22.189594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde18540-c3a4-4462-b431-ad2eb21ebcf0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.194090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c6508-e040-46ed-9b11-b5cebdd2d7c2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.238145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884df8b8-3e89-4ab6-b941-a670beaa2053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.238771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884df8b8-3e89-4ab6-b941-a670beaa2053 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.239049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884df8b8-3e89-4ab6-b941-a670beaa2053 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.239319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884df8b8-3e89-4ab6-b941-a670beaa2053 None admin] PATH_INFO: `/v3/groups/5c4af3860a484ddda332a48e2765d284` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.240680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884df8b8-3e89-4ab6-b941-a670beaa2053 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4af3860a484ddda332a48e2765d284)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:22.243022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884df8b8-3e89-4ab6-b941-a670beaa2053 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:22.243872 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 184/365] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:22 2022] GET /identity/v3/groups/5c4af3860a484ddda332a48e2765d284 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:22.244655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884df8b8-3e89-4ab6-b941-a670beaa2053 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.298953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:22.316600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:22.326066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde18540-c3a4-4462-b431-ad2eb21ebcf0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.368587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5dbe0-ee45-4dc8-b445-a0a84ae53852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.369251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5dbe0-ee45-4dc8-b445-a0a84ae53852 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.369537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5dbe0-ee45-4dc8-b445-a0a84ae53852 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.369632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884df8b8-3e89-4ab6-b941-a670beaa2053 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.369819 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5dbe0-ee45-4dc8-b445-a0a84ae53852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.418330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4832ff7-2d70-448f-a9cd-930c52935854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.419515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4832ff7-2d70-448f-a9cd-930c52935854 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.420019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4832ff7-2d70-448f-a9cd-930c52935854 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.420512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4832ff7-2d70-448f-a9cd-930c52935854 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.432505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4832ff7-2d70-448f-a9cd-930c52935854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f4832ff7-2d70-448f-a9cd-930c52935854 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:22.440484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:22.441369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:22.442228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:22.442228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:22.442228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:22.442228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:22.442228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:22.442228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:22.442228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:22.442228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:22.442228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:22.442228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:22.442228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:22.442228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:22.445018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 185/366] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:22.446130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4832ff7-2d70-448f-a9cd-930c52935854 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.453472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5dbe0-ee45-4dc8-b445-a0a84ae53852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:22.455567 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5dbe0-ee45-4dc8-b445-a0a84ae53852 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:22.551182 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:22.581260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4832ff7-2d70-448f-a9cd-930c52935854 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.587912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 182/367] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:22.588825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5dbe0-ee45-4dc8-b445-a0a84ae53852 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.603383 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:22.630005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ced4b9-1495-4214-a1a1-da2d5268bd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.630661 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ced4b9-1495-4214-a1a1-da2d5268bd50 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.630949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ced4b9-1495-4214-a1a1-da2d5268bd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.631223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ced4b9-1495-4214-a1a1-da2d5268bd50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.707960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5dbe0-ee45-4dc8-b445-a0a84ae53852 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.720846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ced4b9-1495-4214-a1a1-da2d5268bd50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:22.722301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ced4b9-1495-4214-a1a1-da2d5268bd50 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:22.744287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bd9b8-bdac-45fd-a9b3-bc6a1982dba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.744949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bd9b8-bdac-45fd-a9b3-bc6a1982dba0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.745231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bd9b8-bdac-45fd-a9b3-bc6a1982dba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.745510 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bd9b8-bdac-45fd-a9b3-bc6a1982dba0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.746343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bd9b8-bdac-45fd-a9b3-bc6a1982dba0 None admin] RBAC: Adding query filter params (group.id=5c4af3860a484ddda332a48e2765d284, role.id=8a0cfaebc67a497a98459da3f1a82ef9, scope.project.id=2f8281ddbf1345088c2c4475cad30949) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:22.746649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bd9b8-bdac-45fd-a9b3-bc6a1982dba0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:22.748110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bd9b8-bdac-45fd-a9b3-bc6a1982dba0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:22.754081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 183/368] 10.222.0.65 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:22 2022] GET /identity/v3/role_assignments?group.id=5c4af3860a484ddda332a48e2765d284&scope.project.id=2f8281ddbf1345088c2c4475cad30949&role.id=8a0cfaebc67a497a98459da3f1a82ef9 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:22.764057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bd9b8-bdac-45fd-a9b3-bc6a1982dba0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.810695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:22.851866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 186/369] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:22.855168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ced4b9-1495-4214-a1a1-da2d5268bd50 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.880889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:22.882613 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bd9b8-bdac-45fd-a9b3-bc6a1982dba0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.922835 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499c3e7-57cd-4dbe-8b69-0d64417a094c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.923695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499c3e7-57cd-4dbe-8b69-0d64417a094c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.924103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499c3e7-57cd-4dbe-8b69-0d64417a094c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.924459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499c3e7-57cd-4dbe-8b69-0d64417a094c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.925314 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499c3e7-57cd-4dbe-8b69-0d64417a094c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:22.925697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499c3e7-57cd-4dbe-8b69-0d64417a094c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:22.927176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499c3e7-57cd-4dbe-8b69-0d64417a094c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:22.936344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 184/370] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:22.938084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499c3e7-57cd-4dbe-8b69-0d64417a094c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.992910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ced4b9-1495-4214-a1a1-da2d5268bd50 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.030870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a78ad3-872a-4923-a009-75f4580e7d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.031917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a78ad3-872a-4923-a009-75f4580e7d8a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.032211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a78ad3-872a-4923-a009-75f4580e7d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.032492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a78ad3-872a-4923-a009-75f4580e7d8a None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.033815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a78ad3-872a-4923-a009-75f4580e7d8a None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.035253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a78ad3-872a-4923-a009-75f4580e7d8a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.037406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 187/371] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:22 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:23.038198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a78ad3-872a-4923-a009-75f4580e7d8a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.051142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:23.058233 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499c3e7-57cd-4dbe-8b69-0d64417a094c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.111087 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bca4b4-9469-44a4-aeb4-9df8123bf550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.112974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bca4b4-9469-44a4-aeb4-9df8123bf550 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.113698 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bca4b4-9469-44a4-aeb4-9df8123bf550 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.114152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bca4b4-9469-44a4-aeb4-9df8123bf550 None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.116411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bca4b4-9469-44a4-aeb4-9df8123bf550 None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.118601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bca4b4-9469-44a4-aeb4-9df8123bf550 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.121478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 185/372] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:22 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:23.122768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bca4b4-9469-44a4-aeb4-9df8123bf550 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.141626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:23.161194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a78ad3-872a-4923-a009-75f4580e7d8a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.199184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5c37a-fd12-4fdb-89d4-2973dfe1c44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.199882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5c37a-fd12-4fdb-89d4-2973dfe1c44d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.200172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5c37a-fd12-4fdb-89d4-2973dfe1c44d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.200448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5c37a-fd12-4fdb-89d4-2973dfe1c44d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.235285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bca4b4-9469-44a4-aeb4-9df8123bf550 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.271093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b648e-bdd4-482c-9ac5-e7f8f5438e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.271984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b648e-bdd4-482c-9ac5-e7f8f5438e29 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.272264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b648e-bdd4-482c-9ac5-e7f8f5438e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.272537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b648e-bdd4-482c-9ac5-e7f8f5438e29 None admin] PATH_INFO: `/v3/groups/5c4af3860a484ddda332a48e2765d284` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.273230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5c37a-fd12-4fdb-89d4-2973dfe1c44d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.273726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b648e-bdd4-482c-9ac5-e7f8f5438e29 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4af3860a484ddda332a48e2765d284)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.274711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5c37a-fd12-4fdb-89d4-2973dfe1c44d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.275155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b648e-bdd4-482c-9ac5-e7f8f5438e29 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.276824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 186/373] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/groups/5c4af3860a484ddda332a48e2765d284 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:23.277659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b648e-bdd4-482c-9ac5-e7f8f5438e29 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.312669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:23.360584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:23.378171 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b648e-bdd4-482c-9ac5-e7f8f5438e29 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.385956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 188/374] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:23.386831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5c37a-fd12-4fdb-89d4-2973dfe1c44d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.412921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a0ace-f9b6-413d-b61a-a5ce15ab99a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.413779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a0ace-f9b6-413d-b61a-a5ce15ab99a2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.414057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a0ace-f9b6-413d-b61a-a5ce15ab99a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.414332 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a0ace-f9b6-413d-b61a-a5ce15ab99a2 None admin] PATH_INFO: `/v3/groups/5c4af3860a484ddda332a48e2765d284` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.415501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a0ace-f9b6-413d-b61a-a5ce15ab99a2 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4af3860a484ddda332a48e2765d284)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.416978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a0ace-f9b6-413d-b61a-a5ce15ab99a2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.423767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 187/375] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/groups/5c4af3860a484ddda332a48e2765d284 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:23.424594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a0ace-f9b6-413d-b61a-a5ce15ab99a2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.490505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5c37a-fd12-4fdb-89d4-2973dfe1c44d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.528267 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50cda7-0b8a-41d7-943a-220200c1fd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.528720 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a0ace-f9b6-413d-b61a-a5ce15ab99a2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.529165 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50cda7-0b8a-41d7-943a-220200c1fd5e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.529243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50cda7-0b8a-41d7-943a-220200c1fd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.529468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50cda7-0b8a-41d7-943a-220200c1fd5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.530473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50cda7-0b8a-41d7-943a-220200c1fd5e None admin] RBAC: Adding query filter params (group.id=5c4af3860a484ddda332a48e2765d284, role.id=8a0cfaebc67a497a98459da3f1a82ef9, scope.project.id=73330eaa9cf649e283e878927d1e2da5) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:23.530820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50cda7-0b8a-41d7-943a-220200c1fd5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.531076 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:23.533077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50cda7-0b8a-41d7-943a-220200c1fd5e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.539791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 189/376] 10.222.0.65 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/role_assignments?group.id=5c4af3860a484ddda332a48e2765d284&scope.project.id=73330eaa9cf649e283e878927d1e2da5&role.id=8a0cfaebc67a497a98459da3f1a82ef9 => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:23.540612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50cda7-0b8a-41d7-943a-220200c1fd5e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.567966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fb1d8-8aec-45aa-a23c-e85e8c143e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.568840 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fb1d8-8aec-45aa-a23c-e85e8c143e5a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.569118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fb1d8-8aec-45aa-a23c-e85e8c143e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.569390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fb1d8-8aec-45aa-a23c-e85e8c143e5a None admin] PATH_INFO: `/v3/projects/06f70799381a401b8e11df5b00b4192c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.570599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fb1d8-8aec-45aa-a23c-e85e8c143e5a None admin] RBAC: Authorizing `identity:get_project(project_id=06f70799381a401b8e11df5b00b4192c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.572120 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fb1d8-8aec-45aa-a23c-e85e8c143e5a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.574223 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 188/377] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/projects/06f70799381a401b8e11df5b00b4192c => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:23.584808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fb1d8-8aec-45aa-a23c-e85e8c143e5a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.636167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:23.708628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fb1d8-8aec-45aa-a23c-e85e8c143e5a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.749107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f5b27-a26f-4632-8202-fcbc6a9683c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.750903 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f5b27-a26f-4632-8202-fcbc6a9683c4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.751195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f5b27-a26f-4632-8202-fcbc6a9683c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.751477 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f5b27-a26f-4632-8202-fcbc6a9683c4 None admin] PATH_INFO: `/v3/groups/6020625330c247569b92315b4f22b7dd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.757675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50cda7-0b8a-41d7-943a-220200c1fd5e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.758691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f5b27-a26f-4632-8202-fcbc6a9683c4 None admin] RBAC: Authorizing `identity:get_group(group_id=6020625330c247569b92315b4f22b7dd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.760385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f5b27-a26f-4632-8202-fcbc6a9683c4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.762513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 189/378] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/groups/6020625330c247569b92315b4f22b7dd => generated 229 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:23.769450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f5b27-a26f-4632-8202-fcbc6a9683c4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.812151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:23.833548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc74b9-2bcf-4148-8ba0-ca2b2c03887f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.834209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc74b9-2bcf-4148-8ba0-ca2b2c03887f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.834487 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc74b9-2bcf-4148-8ba0-ca2b2c03887f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.834758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc74b9-2bcf-4148-8ba0-ca2b2c03887f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.835590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc74b9-2bcf-4148-8ba0-ca2b2c03887f None admin] RBAC: Adding query filter params (group.id=5c4af3860a484ddda332a48e2765d284, role.id=7d15e336f8034a1fbdef48cfb7c4f717, scope.project.id=73330eaa9cf649e283e878927d1e2da5) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:23.836837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc74b9-2bcf-4148-8ba0-ca2b2c03887f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.838360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc74b9-2bcf-4148-8ba0-ca2b2c03887f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.844549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 190/379] 10.222.0.65 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/role_assignments?group.id=5c4af3860a484ddda332a48e2765d284&scope.project.id=73330eaa9cf649e283e878927d1e2da5&role.id=7d15e336f8034a1fbdef48cfb7c4f717 => generated 260 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:23.888695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f5b27-a26f-4632-8202-fcbc6a9683c4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.928784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd1a35-1f0f-40bd-a71f-77847615360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.929480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd1a35-1f0f-40bd-a71f-77847615360c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.929768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd1a35-1f0f-40bd-a71f-77847615360c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.930050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd1a35-1f0f-40bd-a71f-77847615360c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.930930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd1a35-1f0f-40bd-a71f-77847615360c None admin] RBAC: Adding query filter params (group.id=6020625330c247569b92315b4f22b7dd, role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=06f70799381a401b8e11df5b00b4192c) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:23.931253 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd1a35-1f0f-40bd-a71f-77847615360c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.932924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd1a35-1f0f-40bd-a71f-77847615360c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.939697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 190/380] 10.222.0.65 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/role_assignments?group.id=6020625330c247569b92315b4f22b7dd&scope.project.id=06f70799381a401b8e11df5b00b4192c&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:24.190441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc941f5-e431-4c3a-bb7d-21bb776149b1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.190732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc941f5-e431-4c3a-bb7d-21bb776149b1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.190931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc941f5-e431-4c3a-bb7d-21bb776149b1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.192609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 191/381] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:24.197862 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea880f2e-010e-495f-9cd0-f826ba27a630 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.198158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea880f2e-010e-495f-9cd0-f826ba27a630 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.198466 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea880f2e-010e-495f-9cd0-f826ba27a630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.219625 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ea880f2e-010e-495f-9cd0-f826ba27a630 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:24.220853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ea880f2e-010e-495f-9cd0-f826ba27a630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:24.311478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 191/382] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:24.355231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc941f5-e431-4c3a-bb7d-21bb776149b1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:24.509355 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc941f5-e431-4c3a-bb7d-21bb776149b1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:24.523286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:24.581257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a0e85-738d-4ef0-9684-8193fa68c0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:24.582068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a0e85-738d-4ef0-9684-8193fa68c0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.582347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a0e85-738d-4ef0-9684-8193fa68c0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.582621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a0e85-738d-4ef0-9684-8193fa68c0f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.609369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a0e85-738d-4ef0-9684-8193fa68c0f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-195a0e85-738d-4ef0-9684-8193fa68c0f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:24.617109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:24.617993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:24.618788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 192/383] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:24.624719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea880f2e-010e-495f-9cd0-f826ba27a630 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:24.661003 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:24.726788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea880f2e-010e-495f-9cd0-f826ba27a630 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:24.778653 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37481f-bc19-479d-82de-6b519a4b9f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:24.779332 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37481f-bc19-479d-82de-6b519a4b9f9e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.779611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37481f-bc19-479d-82de-6b519a4b9f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.779910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37481f-bc19-479d-82de-6b519a4b9f9e None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.781189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37481f-bc19-479d-82de-6b519a4b9f9e None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:24.781497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37481f-bc19-479d-82de-6b519a4b9f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:24.782869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37481f-bc19-479d-82de-6b519a4b9f9e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:24.795315 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 192/384] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:24.801332 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a0e85-738d-4ef0-9684-8193fa68c0f9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:24.933295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a0e85-738d-4ef0-9684-8193fa68c0f9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:24.981569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c09d6-5b44-4fe2-870c-43be9c2b8274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:24.982427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c09d6-5b44-4fe2-870c-43be9c2b8274 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.982708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c09d6-5b44-4fe2-870c-43be9c2b8274 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.982981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c09d6-5b44-4fe2-870c-43be9c2b8274 None admin] PATH_INFO: `/v3/groups/5c4af3860a484ddda332a48e2765d284` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.984455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c09d6-5b44-4fe2-870c-43be9c2b8274 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4af3860a484ddda332a48e2765d284)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:24.985887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c09d6-5b44-4fe2-870c-43be9c2b8274 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:24.990560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 193/385] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:24 2022] GET /identity/v3/groups/5c4af3860a484ddda332a48e2765d284 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:24.992664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45caae-df01-4e69-a5c4-4669a2852a82 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.992961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45caae-df01-4e69-a5c4-4669a2852a82 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.993244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45caae-df01-4e69-a5c4-4669a2852a82 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.993344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c09d6-5b44-4fe2-870c-43be9c2b8274 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:24.994838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 193/386] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:24.999753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed483705-0177-44ab-a83d-3cdad6322c89 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.000046 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed483705-0177-44ab-a83d-3cdad6322c89 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.000328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed483705-0177-44ab-a83d-3cdad6322c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.017996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ed483705-0177-44ab-a83d-3cdad6322c89 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:25.019455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ed483705-0177-44ab-a83d-3cdad6322c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:25.034083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:25.104411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c09d6-5b44-4fe2-870c-43be9c2b8274 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.148276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b203a-f01f-4e99-9589-37ffa6306c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.149253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b203a-f01f-4e99-9589-37ffa6306c60 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.149533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b203a-f01f-4e99-9589-37ffa6306c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.149811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b203a-f01f-4e99-9589-37ffa6306c60 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.156778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b203a-f01f-4e99-9589-37ffa6306c60 None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:25.158252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b203a-f01f-4e99-9589-37ffa6306c60 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:25.160465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 194/387] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:24 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:25.166023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b203a-f01f-4e99-9589-37ffa6306c60 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.173679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 194/388] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:25.196292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed483705-0177-44ab-a83d-3cdad6322c89 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.221359 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:25.316326 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed483705-0177-44ab-a83d-3cdad6322c89 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.354782 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84992b-6d22-40f1-bc59-bf875cd60ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.355878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84992b-6d22-40f1-bc59-bf875cd60ecc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.356622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84992b-6d22-40f1-bc59-bf875cd60ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.357044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84992b-6d22-40f1-bc59-bf875cd60ecc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.369026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84992b-6d22-40f1-bc59-bf875cd60ecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0d84992b-6d22-40f1-bc59-bf875cd60ecc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:25.379128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:25.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:25.381285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:25.381285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:25.381285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:25.381285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:25.381285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:25.381285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:25.381285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:25.381285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:25.381285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:25.381285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:25.381285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:25.381285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:25.387953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 195/389] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:25.390591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84992b-6d22-40f1-bc59-bf875cd60ecc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.397546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b203a-f01f-4e99-9589-37ffa6306c60 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.424032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:25.440089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba043d2-c416-41f3-a2c2-0606ed34a514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.442581 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba043d2-c416-41f3-a2c2-0606ed34a514 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.443488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba043d2-c416-41f3-a2c2-0606ed34a514 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.444573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba043d2-c416-41f3-a2c2-0606ed34a514 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949/groups/5c4af3860a484ddda332a48e2765d284/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.449792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba043d2-c416-41f3-a2c2-0606ed34a514 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f8281ddbf1345088c2c4475cad30949, group_id=5c4af3860a484ddda332a48e2765d284, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:25.451431 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba043d2-c416-41f3-a2c2-0606ed34a514 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:25.476400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 195/390] 10.222.0.65 () {58 vars in 1442 bytes} [Thu Jun 23 10:49:25 2022] PUT /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949/groups/5c4af3860a484ddda332a48e2765d284/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:49:25.531627 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:25.552663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84992b-6d22-40f1-bc59-bf875cd60ecc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.594905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db5026-05cd-4c82-aee1-7916dfc8798d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.596017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db5026-05cd-4c82-aee1-7916dfc8798d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.596502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db5026-05cd-4c82-aee1-7916dfc8798d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.596971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db5026-05cd-4c82-aee1-7916dfc8798d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.597997 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db5026-05cd-4c82-aee1-7916dfc8798d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:25.598542 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db5026-05cd-4c82-aee1-7916dfc8798d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:25.600204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db5026-05cd-4c82-aee1-7916dfc8798d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:25.616142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 196/391] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:25.622069 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba043d2-c416-41f3-a2c2-0606ed34a514 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.742181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba043d2-c416-41f3-a2c2-0606ed34a514 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.793447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e5f8d-ea9f-409d-82ab-50a7bc82d49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.794308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e5f8d-ea9f-409d-82ab-50a7bc82d49b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.794588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e5f8d-ea9f-409d-82ab-50a7bc82d49b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.794862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e5f8d-ea9f-409d-82ab-50a7bc82d49b None admin] PATH_INFO: `/v3/groups/5c4af3860a484ddda332a48e2765d284` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.800339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e5f8d-ea9f-409d-82ab-50a7bc82d49b None admin] RBAC: Authorizing `identity:get_group(group_id=5c4af3860a484ddda332a48e2765d284)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:25.801762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e5f8d-ea9f-409d-82ab-50a7bc82d49b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:25.803545 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 196/392] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:25 2022] GET /identity/v3/groups/5c4af3860a484ddda332a48e2765d284 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:25.809802 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db5026-05cd-4c82-aee1-7916dfc8798d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.957820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db5026-05cd-4c82-aee1-7916dfc8798d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:26.006266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676efb2-6861-44ae-856f-f0138c078b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:26.007152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676efb2-6861-44ae-856f-f0138c078b5f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.007436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676efb2-6861-44ae-856f-f0138c078b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.007752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676efb2-6861-44ae-856f-f0138c078b5f None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.009119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676efb2-6861-44ae-856f-f0138c078b5f None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:26.010616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676efb2-6861-44ae-856f-f0138c078b5f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:26.012329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 197/393] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:25 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:26.018177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e5f8d-ea9f-409d-82ab-50a7bc82d49b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:26.086904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:26.190651 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e5f8d-ea9f-409d-82ab-50a7bc82d49b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:26.225570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd2080-3624-4505-a972-43805804f51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:26.226786 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2080-3624-4505-a972-43805804f51c None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.227286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2080-3624-4505-a972-43805804f51c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.227784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2080-3624-4505-a972-43805804f51c None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949/groups/5c4af3860a484ddda332a48e2765d284/roles/8a0cfaebc67a497a98459da3f1a82ef9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.230175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd2080-3624-4505-a972-43805804f51c None admin] RBAC: Authorizing `identity:create_grant(project_id=2f8281ddbf1345088c2c4475cad30949, group_id=5c4af3860a484ddda332a48e2765d284, role_id=8a0cfaebc67a497a98459da3f1a82ef9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:26.231838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd2080-3624-4505-a972-43805804f51c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:26.248382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 197/394] 10.222.0.65 () {58 vars in 1442 bytes} [Thu Jun 23 10:49:26 2022] PUT /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949/groups/5c4af3860a484ddda332a48e2765d284/roles/8a0cfaebc67a497a98459da3f1a82ef9 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:26.372038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd97df-5bb1-4f80-8fc1-2b2718fe8333 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.372427 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd97df-5bb1-4f80-8fc1-2b2718fe8333 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.372609 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd97df-5bb1-4f80-8fc1-2b2718fe8333 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.374292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 198/395] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:26.379737 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3d082-ea47-4b28-a1af-30d532a2e8da None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.380026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3d082-ea47-4b28-a1af-30d532a2e8da None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.380313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3d082-ea47-4b28-a1af-30d532a2e8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.398262 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3ee3d082-ea47-4b28-a1af-30d532a2e8da None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:26.399465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3ee3d082-ea47-4b28-a1af-30d532a2e8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:26.499765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 198/396] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:26.515297 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81237c2e-f38d-468f-8567-be6596c029ff None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.515942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81237c2e-f38d-468f-8567-be6596c029ff None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.516432 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81237c2e-f38d-468f-8567-be6596c029ff None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.518226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 199/397] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:26.524882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e76c9-13b5-4bc1-99ef-df80f89e336b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.526184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81237c2e-f38d-468f-8567-be6596c029ff None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:26.527461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e76c9-13b5-4bc1-99ef-df80f89e336b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.527978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e76c9-13b5-4bc1-99ef-df80f89e336b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.549573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-202e76c9-13b5-4bc1-99ef-df80f89e336b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:26.551174 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-202e76c9-13b5-4bc1-99ef-df80f89e336b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:26.552393 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:26.574582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:26.642303 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 199/398] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:26.651674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81237c2e-f38d-468f-8567-be6596c029ff None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:26.651819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79edc2-c671-4858-84fc-a43100b209e0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.651819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79edc2-c671-4858-84fc-a43100b209e0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.651819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79edc2-c671-4858-84fc-a43100b209e0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.654723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 200/399] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:26.660600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b999d-c014-4983-88b4-53b788790abb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.661095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b999d-c014-4983-88b4-53b788790abb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.661555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b999d-c014-4983-88b4-53b788790abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.687784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-743b999d-c014-4983-88b4-53b788790abb None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:26.689334 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-743b999d-c014-4983-88b4-53b788790abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:26.695204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beddfe2-7ad8-4491-b2b9-d56063571cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:26.696259 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beddfe2-7ad8-4491-b2b9-d56063571cfe None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.696749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beddfe2-7ad8-4491-b2b9-d56063571cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.697208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beddfe2-7ad8-4491-b2b9-d56063571cfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.708737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beddfe2-7ad8-4491-b2b9-d56063571cfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7beddfe2-7ad8-4491-b2b9-d56063571cfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:26.716865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:26.717779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:26.719192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:26.719192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:26.719192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:26.719192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:26.719192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:26.719192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:26.719192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:26.719192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:26.719192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:26.719192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:26.719192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:26.719192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:26.721719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 200/400] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:26.722781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beddfe2-7ad8-4491-b2b9-d56063571cfe None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:26.781373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 201/401] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:26.782441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b999d-c014-4983-88b4-53b788790abb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:26.804077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:26.845636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beddfe2-7ad8-4491-b2b9-d56063571cfe None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:26.890539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a774c71-041b-420f-873d-0bcc7a902ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:26.891523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a774c71-041b-420f-873d-0bcc7a902ccf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.891912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a774c71-041b-420f-873d-0bcc7a902ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.892303 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a774c71-041b-420f-873d-0bcc7a902ccf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.894630 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:26.905659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a774c71-041b-420f-873d-0bcc7a902ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1a774c71-041b-420f-873d-0bcc7a902ccf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:26.915066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:26.916290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:26.917291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:26.917291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:26.917291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:26.917291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:26.917291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:26.917291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:26.917291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:26.917291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:26.917291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:26.917291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:26.917291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:26.917291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:26.918311 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b999d-c014-4983-88b4-53b788790abb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:26.919171 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 201/402] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:26.920224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a774c71-041b-420f-873d-0bcc7a902ccf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:26.963981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5756d-f301-4140-be7c-8f84acdae33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:26.965435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5756d-f301-4140-be7c-8f84acdae33b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.965881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5756d-f301-4140-be7c-8f84acdae33b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.966310 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5756d-f301-4140-be7c-8f84acdae33b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.967674 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5756d-f301-4140-be7c-8f84acdae33b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:26.968172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5756d-f301-4140-be7c-8f84acdae33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:26.970423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5756d-f301-4140-be7c-8f84acdae33b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:26.984544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 202/403] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:26.985850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5756d-f301-4140-be7c-8f84acdae33b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:27.045204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a774c71-041b-420f-873d-0bcc7a902ccf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:27.078388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:27.086214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb34c7-a568-4468-8980-81035571a0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:27.087975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb34c7-a568-4468-8980-81035571a0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:27.088481 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb34c7-a568-4468-8980-81035571a0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:27.089232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb34c7-a568-4468-8980-81035571a0f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:27.102286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5756d-f301-4140-be7c-8f84acdae33b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:27.110038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb34c7-a568-4468-8980-81035571a0f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-adeb34c7-a568-4468-8980-81035571a0f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:27.118761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:27.119673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:27.120480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:27.120480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:27.120480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:27.120480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:27.120480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:27.120480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:27.120480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:27.120480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:27.120480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:27.120480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:27.120480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:27.120480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:27.122851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 202/404] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:27.123822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb34c7-a568-4468-8980-81035571a0f9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:27.147276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c53e1-48c6-4273-a9be-1357ad9b2d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:27.148642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c53e1-48c6-4273-a9be-1357ad9b2d30 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:27.149353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c53e1-48c6-4273-a9be-1357ad9b2d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:27.149781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c53e1-48c6-4273-a9be-1357ad9b2d30 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:27.150860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c53e1-48c6-4273-a9be-1357ad9b2d30 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:27.151327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c53e1-48c6-4273-a9be-1357ad9b2d30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:27.153185 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c53e1-48c6-4273-a9be-1357ad9b2d30 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:27.168617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 203/405] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:27.168790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c53e1-48c6-4273-a9be-1357ad9b2d30 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:27.181334 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:27.247398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb34c7-a568-4468-8980-81035571a0f9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:27.276211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c53e1-48c6-4273-a9be-1357ad9b2d30 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:27.284593 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf31d58-0f03-41a0-be0a-9753ca46b763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:27.285833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf31d58-0f03-41a0-be0a-9753ca46b763 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:27.286330 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf31d58-0f03-41a0-be0a-9753ca46b763 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:27.286814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf31d58-0f03-41a0-be0a-9753ca46b763 None admin] PATH_INFO: `/v3/groups/6020625330c247569b92315b4f22b7dd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:27.288294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf31d58-0f03-41a0-be0a-9753ca46b763 None admin] RBAC: Authorizing `identity:get_group(group_id=6020625330c247569b92315b4f22b7dd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:27.290086 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf31d58-0f03-41a0-be0a-9753ca46b763 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:27.292358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 203/406] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:27 2022] GET /identity/v3/groups/6020625330c247569b92315b4f22b7dd => generated 229 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:27.293501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf31d58-0f03-41a0-be0a-9753ca46b763 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:27.313340 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26de5e-4b87-47f4-b86e-444dd3005871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:27.313987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26de5e-4b87-47f4-b86e-444dd3005871 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:27.314270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26de5e-4b87-47f4-b86e-444dd3005871 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:27.314543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26de5e-4b87-47f4-b86e-444dd3005871 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:27.315365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26de5e-4b87-47f4-b86e-444dd3005871 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:27.315622 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26de5e-4b87-47f4-b86e-444dd3005871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:27.317107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26de5e-4b87-47f4-b86e-444dd3005871 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:27.330908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 204/407] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:27.331814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26de5e-4b87-47f4-b86e-444dd3005871 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:27.361231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:27.406447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf31d58-0f03-41a0-be0a-9753ca46b763 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:27.408796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:27.443001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26de5e-4b87-47f4-b86e-444dd3005871 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:27.447092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbb185-beac-472e-b802-03113afcb935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:27.448109 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbb185-beac-472e-b802-03113afcb935 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:27.448607 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbb185-beac-472e-b802-03113afcb935 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:27.448985 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbb185-beac-472e-b802-03113afcb935 None admin] PATH_INFO: `/v3/groups/5c4af3860a484ddda332a48e2765d284` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:27.450330 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbb185-beac-472e-b802-03113afcb935 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4af3860a484ddda332a48e2765d284)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:27.452107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbb185-beac-472e-b802-03113afcb935 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:27.458404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 204/408] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:27 2022] GET /identity/v3/groups/5c4af3860a484ddda332a48e2765d284 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:27.459545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbb185-beac-472e-b802-03113afcb935 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:27.480520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75db822-0522-46a9-9a1c-6ffb4b3d09fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:27.481704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75db822-0522-46a9-9a1c-6ffb4b3d09fd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:27.482157 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75db822-0522-46a9-9a1c-6ffb4b3d09fd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:27.482602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75db822-0522-46a9-9a1c-6ffb4b3d09fd None admin] PATH_INFO: `/v3/projects/06f70799381a401b8e11df5b00b4192c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:27.484812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75db822-0522-46a9-9a1c-6ffb4b3d09fd None admin] RBAC: Authorizing `identity:get_project(project_id=06f70799381a401b8e11df5b00b4192c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:27.486428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75db822-0522-46a9-9a1c-6ffb4b3d09fd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:27.488704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 205/409] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:27 2022] GET /identity/v3/projects/06f70799381a401b8e11df5b00b4192c => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:27.489778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75db822-0522-46a9-9a1c-6ffb4b3d09fd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:27.568573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbb185-beac-472e-b802-03113afcb935 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:27.597954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:27.611487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1fc5e-b21b-4b9b-98a1-fe5881d38dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:27.612395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1fc5e-b21b-4b9b-98a1-fe5881d38dcb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:27.612681 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1fc5e-b21b-4b9b-98a1-fe5881d38dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:27.613015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1fc5e-b21b-4b9b-98a1-fe5881d38dcb None admin] PATH_INFO: `/v3/groups/5c4af3860a484ddda332a48e2765d284` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:27.614729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1fc5e-b21b-4b9b-98a1-fe5881d38dcb None admin] RBAC: Authorizing `identity:get_group(group_id=5c4af3860a484ddda332a48e2765d284)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:27.616489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1fc5e-b21b-4b9b-98a1-fe5881d38dcb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:27.618946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 205/410] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:27 2022] GET /identity/v3/groups/5c4af3860a484ddda332a48e2765d284 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:27.619868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1fc5e-b21b-4b9b-98a1-fe5881d38dcb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:27.627441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75db822-0522-46a9-9a1c-6ffb4b3d09fd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:27.661906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5ba57-b827-414a-b7de-b46208d5657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:27.662904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5ba57-b827-414a-b7de-b46208d5657f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:27.663253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5ba57-b827-414a-b7de-b46208d5657f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:27.663622 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5ba57-b827-414a-b7de-b46208d5657f None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:27.664911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5ba57-b827-414a-b7de-b46208d5657f None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:27.666470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5ba57-b827-414a-b7de-b46208d5657f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:27.673736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 206/411] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:27 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:27.674754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5ba57-b827-414a-b7de-b46208d5657f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:27.683989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:27.749952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1fc5e-b21b-4b9b-98a1-fe5881d38dcb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:27.784619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5ba57-b827-414a-b7de-b46208d5657f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:27.795577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916449a-eb50-4abb-9318-8cfb90ac7c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:27.797453 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916449a-eb50-4abb-9318-8cfb90ac7c04 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:27.797757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916449a-eb50-4abb-9318-8cfb90ac7c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:27.798039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916449a-eb50-4abb-9318-8cfb90ac7c04 None admin] PATH_INFO: `/v3/projects/06f70799381a401b8e11df5b00b4192c/groups/6020625330c247569b92315b4f22b7dd/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:27.800608 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916449a-eb50-4abb-9318-8cfb90ac7c04 None admin] RBAC: Authorizing `identity:create_grant(project_id=06f70799381a401b8e11df5b00b4192c, group_id=6020625330c247569b92315b4f22b7dd, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:27.802876 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916449a-eb50-4abb-9318-8cfb90ac7c04 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:27.823828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 206/412] 10.222.0.65 () {58 vars in 1442 bytes} [Thu Jun 23 10:49:27 2022] PUT /identity/v3/projects/06f70799381a401b8e11df5b00b4192c/groups/6020625330c247569b92315b4f22b7dd/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:27.824926 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916449a-eb50-4abb-9318-8cfb90ac7c04 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:27.835231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b90a99-7634-4ef3-9a65-b72e74e795bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:27.836172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b90a99-7634-4ef3-9a65-b72e74e795bb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:27.836459 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b90a99-7634-4ef3-9a65-b72e74e795bb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:27.836739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b90a99-7634-4ef3-9a65-b72e74e795bb None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:27.840519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b90a99-7634-4ef3-9a65-b72e74e795bb None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:27.844476 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b90a99-7634-4ef3-9a65-b72e74e795bb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:27.850490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 207/413] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:27 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:27.857056 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b90a99-7634-4ef3-9a65-b72e74e795bb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:27.913351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:27.961459 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:27.968541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916449a-eb50-4abb-9318-8cfb90ac7c04 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.001598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b90a99-7634-4ef3-9a65-b72e74e795bb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.010155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd769ab2-1518-4949-b874-2a23cd0b30c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.011292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd769ab2-1518-4949-b874-2a23cd0b30c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.011777 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd769ab2-1518-4949-b874-2a23cd0b30c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.012202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd769ab2-1518-4949-b874-2a23cd0b30c1 None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5/groups/5c4af3860a484ddda332a48e2765d284/roles/8a0cfaebc67a497a98459da3f1a82ef9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.015295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd769ab2-1518-4949-b874-2a23cd0b30c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=73330eaa9cf649e283e878927d1e2da5, group_id=5c4af3860a484ddda332a48e2765d284, role_id=8a0cfaebc67a497a98459da3f1a82ef9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:28.017009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd769ab2-1518-4949-b874-2a23cd0b30c1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:28.032718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 207/414] 10.222.0.65 () {58 vars in 1442 bytes} [Thu Jun 23 10:49:27 2022] PUT /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5/groups/5c4af3860a484ddda332a48e2765d284/roles/8a0cfaebc67a497a98459da3f1a82ef9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:28.034366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1c45e-228c-4f4f-9ee4-277304df41e1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.034810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1c45e-228c-4f4f-9ee4-277304df41e1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.035277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1c45e-228c-4f4f-9ee4-277304df41e1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.037179 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 208/415] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:28.042837 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629f691-dafd-46b2-871c-dc72f56fc101 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.043501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629f691-dafd-46b2-871c-dc72f56fc101 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.044054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629f691-dafd-46b2-871c-dc72f56fc101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.048722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3ba13-c0ff-4756-a4d1-8a15009769bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.049633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3ba13-c0ff-4756-a4d1-8a15009769bf None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.050144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3ba13-c0ff-4756-a4d1-8a15009769bf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.050345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3ba13-c0ff-4756-a4d1-8a15009769bf None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5/groups/5c4af3860a484ddda332a48e2765d284/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.055649 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3ba13-c0ff-4756-a4d1-8a15009769bf None admin] RBAC: Authorizing `identity:create_grant(project_id=73330eaa9cf649e283e878927d1e2da5, group_id=5c4af3860a484ddda332a48e2765d284, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:28.057134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3ba13-c0ff-4756-a4d1-8a15009769bf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:28.064033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7629f691-dafd-46b2-871c-dc72f56fc101 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:28.065567 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7629f691-dafd-46b2-871c-dc72f56fc101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:28.101740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 208/416] 10.222.0.65 () {58 vars in 1442 bytes} [Thu Jun 23 10:49:27 2022] PUT /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5/groups/5c4af3860a484ddda332a48e2765d284/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:49:28.147359 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 209/417] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:28.188336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3ba13-c0ff-4756-a4d1-8a15009769bf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.312930 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3ba13-c0ff-4756-a4d1-8a15009769bf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.351250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec6555-f1c3-4856-bc7f-3d84b7b63fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.352022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec6555-f1c3-4856-bc7f-3d84b7b63fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.352372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec6555-f1c3-4856-bc7f-3d84b7b63fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.352657 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec6555-f1c3-4856-bc7f-3d84b7b63fd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.354775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:28.365865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec6555-f1c3-4856-bc7f-3d84b7b63fd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-64ec6555-f1c3-4856-bc7f-3d84b7b63fd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:28.375803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:28.376697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:28.377501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 209/418] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:28.384430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629f691-dafd-46b2-871c-dc72f56fc101 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.440981 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:28.509834 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629f691-dafd-46b2-871c-dc72f56fc101 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.540897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d56b1-109e-4bb6-b3d5-ee17a28d1e10 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.541197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d56b1-109e-4bb6-b3d5-ee17a28d1e10 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.541441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d56b1-109e-4bb6-b3d5-ee17a28d1e10 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.543764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 210/419] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:28.548194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c49cd-06c3-4360-98dc-f47bdf97b2f0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.548482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c49cd-06c3-4360-98dc-f47bdf97b2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.548756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c49cd-06c3-4360-98dc-f47bdf97b2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.560411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29600c65-af17-4afb-a40b-c1b3d4988357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.561111 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29600c65-af17-4afb-a40b-c1b3d4988357 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.561394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29600c65-af17-4afb-a40b-c1b3d4988357 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.562211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29600c65-af17-4afb-a40b-c1b3d4988357 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.562980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29600c65-af17-4afb-a40b-c1b3d4988357 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:28.563290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29600c65-af17-4afb-a40b-c1b3d4988357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:28.564708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29600c65-af17-4afb-a40b-c1b3d4988357 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:28.574634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 210/420] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:28.580503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ee5c49cd-06c3-4360-98dc-f47bdf97b2f0 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:28.580765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29600c65-af17-4afb-a40b-c1b3d4988357 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.581708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ee5c49cd-06c3-4360-98dc-f47bdf97b2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:28.692455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29600c65-af17-4afb-a40b-c1b3d4988357 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.702215 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:28.735901 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b0087-35b9-4e22-9e45-742d99698e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.736578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b0087-35b9-4e22-9e45-742d99698e04 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.737214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b0087-35b9-4e22-9e45-742d99698e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.737503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b0087-35b9-4e22-9e45-742d99698e04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.759782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 211/421] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:28.808587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c49cd-06c3-4360-98dc-f47bdf97b2f0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.841491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b0087-35b9-4e22-9e45-742d99698e04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:28.843009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b0087-35b9-4e22-9e45-742d99698e04 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:28.936197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c49cd-06c3-4360-98dc-f47bdf97b2f0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.955646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 211/422] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:28.960995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b0087-35b9-4e22-9e45-742d99698e04 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.961518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:28.972039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:28.982896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6aa667-2db9-4cbf-aad8-a3594bfdbb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.984566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aa667-2db9-4cbf-aad8-a3594bfdbb4d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.984852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aa667-2db9-4cbf-aad8-a3594bfdbb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.985135 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aa667-2db9-4cbf-aad8-a3594bfdbb4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.009020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6aa667-2db9-4cbf-aad8-a3594bfdbb4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4d6aa667-2db9-4cbf-aad8-a3594bfdbb4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:29.020819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:29.021769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:29.022552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 212/423] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:29.028836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6aa667-2db9-4cbf-aad8-a3594bfdbb4d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:29.068532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b0087-35b9-4e22-9e45-742d99698e04 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.115621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29c22a-366c-4e66-997f-88e616f483ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.116770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29c22a-366c-4e66-997f-88e616f483ea None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.119276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29c22a-366c-4e66-997f-88e616f483ea None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.119750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29c22a-366c-4e66-997f-88e616f483ea None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.122605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29c22a-366c-4e66-997f-88e616f483ea None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:29.125259 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29c22a-366c-4e66-997f-88e616f483ea None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:29.131096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 212/424] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:29.136642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29c22a-366c-4e66-997f-88e616f483ea None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:29.239480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6aa667-2db9-4cbf-aad8-a3594bfdbb4d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.282109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0514785-96dc-4fd7-bf98-932ec0e54b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.283008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0514785-96dc-4fd7-bf98-932ec0e54b63 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.283469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0514785-96dc-4fd7-bf98-932ec0e54b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.283916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0514785-96dc-4fd7-bf98-932ec0e54b63 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.284812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0514785-96dc-4fd7-bf98-932ec0e54b63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:29.285269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0514785-96dc-4fd7-bf98-932ec0e54b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:29.287037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0514785-96dc-4fd7-bf98-932ec0e54b63 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:29.311984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 213/425] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:29.327555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0514785-96dc-4fd7-bf98-932ec0e54b63 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:29.328062 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:29.409642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29c22a-366c-4e66-997f-88e616f483ea None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.415988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:29.440220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0514785-96dc-4fd7-bf98-932ec0e54b63 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.478557 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75613ebc-16af-4143-9ba3-8812fe8dfc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.479448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75613ebc-16af-4143-9ba3-8812fe8dfc03 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.479910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75613ebc-16af-4143-9ba3-8812fe8dfc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.480333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75613ebc-16af-4143-9ba3-8812fe8dfc03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.526682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeb11b-417a-47ac-8120-8e36290c592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.527897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeb11b-417a-47ac-8120-8e36290c592b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.528338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeb11b-417a-47ac-8120-8e36290c592b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.529428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeb11b-417a-47ac-8120-8e36290c592b None admin] PATH_INFO: `/v3/groups/5c4af3860a484ddda332a48e2765d284` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.531564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeb11b-417a-47ac-8120-8e36290c592b None admin] RBAC: Authorizing `identity:get_group(group_id=5c4af3860a484ddda332a48e2765d284)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:29.533808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeb11b-417a-47ac-8120-8e36290c592b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:29.536587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 213/426] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:29 2022] GET /identity/v3/groups/5c4af3860a484ddda332a48e2765d284 => generated 226 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:29.544841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeb11b-417a-47ac-8120-8e36290c592b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:29.563488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75613ebc-16af-4143-9ba3-8812fe8dfc03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:29.565175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75613ebc-16af-4143-9ba3-8812fe8dfc03 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:29.615894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:29.672444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 214/427] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:29.683175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75613ebc-16af-4143-9ba3-8812fe8dfc03 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:29.726719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeb11b-417a-47ac-8120-8e36290c592b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.763206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386a44a-9076-4e3b-87f1-5294923d33e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.764236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386a44a-9076-4e3b-87f1-5294923d33e5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.765334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386a44a-9076-4e3b-87f1-5294923d33e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.765815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386a44a-9076-4e3b-87f1-5294923d33e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.766860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386a44a-9076-4e3b-87f1-5294923d33e5 None admin] RBAC: Adding query filter params (group.id=5c4af3860a484ddda332a48e2765d284, role.id=7d15e336f8034a1fbdef48cfb7c4f717, scope.project.id=2f8281ddbf1345088c2c4475cad30949) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:29.767400 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386a44a-9076-4e3b-87f1-5294923d33e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:29.769064 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386a44a-9076-4e3b-87f1-5294923d33e5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:29.775772 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 214/428] 10.222.0.65 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:29 2022] GET /identity/v3/role_assignments?group.id=5c4af3860a484ddda332a48e2765d284&scope.project.id=2f8281ddbf1345088c2c4475cad30949&role.id=7d15e336f8034a1fbdef48cfb7c4f717 => generated 611 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:29.867113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75613ebc-16af-4143-9ba3-8812fe8dfc03 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.923415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1333e-7492-4cea-9b56-ad49b85efcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.924788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1333e-7492-4cea-9b56-ad49b85efcf9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.924788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1333e-7492-4cea-9b56-ad49b85efcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.925053 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1333e-7492-4cea-9b56-ad49b85efcf9 None admin] PATH_INFO: `/v3/projects/2f8281ddbf1345088c2c4475cad30949` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.926076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1333e-7492-4cea-9b56-ad49b85efcf9 None admin] RBAC: Authorizing `identity:get_project(project_id=2f8281ddbf1345088c2c4475cad30949)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:29.927512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1333e-7492-4cea-9b56-ad49b85efcf9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:29.929237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 215/429] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:29 2022] GET /identity/v3/projects/2f8281ddbf1345088c2c4475cad30949 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:29.935337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386a44a-9076-4e3b-87f1-5294923d33e5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:29.955953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:30.070965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386a44a-9076-4e3b-87f1-5294923d33e5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:30.124198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469ed68-f02f-40f1-a356-149c5999ceb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:30.127608 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469ed68-f02f-40f1-a356-149c5999ceb2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.128199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469ed68-f02f-40f1-a356-149c5999ceb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.128693 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469ed68-f02f-40f1-a356-149c5999ceb2 None admin] PATH_INFO: `/v3/groups/5c4af3860a484ddda332a48e2765d284` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.130191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469ed68-f02f-40f1-a356-149c5999ceb2 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4af3860a484ddda332a48e2765d284)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:30.131919 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469ed68-f02f-40f1-a356-149c5999ceb2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:30.134084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 215/430] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:29 2022] GET /identity/v3/groups/5c4af3860a484ddda332a48e2765d284 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:30.143764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1333e-7492-4cea-9b56-ad49b85efcf9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:30.164007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:30.178130 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff70ba0-3758-42de-9fd0-83c805bd1fa2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.178507 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff70ba0-3758-42de-9fd0-83c805bd1fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.178691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff70ba0-3758-42de-9fd0-83c805bd1fa2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.180372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 216/431] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:30.188364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25812b20-9adf-4753-8dfe-15f367576e8d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.188654 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25812b20-9adf-4753-8dfe-15f367576e8d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.188939 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25812b20-9adf-4753-8dfe-15f367576e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.207829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-25812b20-9adf-4753-8dfe-15f367576e8d None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:30.209046 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-25812b20-9adf-4753-8dfe-15f367576e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:30.258853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1333e-7492-4cea-9b56-ad49b85efcf9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:30.290942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 217/432] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:30.300526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c5bf7-b337-4267-8eb6-71d47d431631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:30.301371 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c5bf7-b337-4267-8eb6-71d47d431631 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.301795 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c5bf7-b337-4267-8eb6-71d47d431631 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.302231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c5bf7-b337-4267-8eb6-71d47d431631 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.303198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c5bf7-b337-4267-8eb6-71d47d431631 None admin] RBAC: Adding query filter params (group.id=5c4af3860a484ddda332a48e2765d284, role.id=8a0cfaebc67a497a98459da3f1a82ef9, scope.project.id=2f8281ddbf1345088c2c4475cad30949) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:30.303733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c5bf7-b337-4267-8eb6-71d47d431631 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:30.305285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c5bf7-b337-4267-8eb6-71d47d431631 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:30.312899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 216/433] 10.222.0.65 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:30 2022] GET /identity/v3/role_assignments?group.id=5c4af3860a484ddda332a48e2765d284&scope.project.id=2f8281ddbf1345088c2c4475cad30949&role.id=8a0cfaebc67a497a98459da3f1a82ef9 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:30.323697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25812b20-9adf-4753-8dfe-15f367576e8d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:30.336072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:30.438432 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25812b20-9adf-4753-8dfe-15f367576e8d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:30.480070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299201cd-f79f-4120-9a4f-daf96ff0756d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:30.480892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299201cd-f79f-4120-9a4f-daf96ff0756d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.481179 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299201cd-f79f-4120-9a4f-daf96ff0756d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.481459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299201cd-f79f-4120-9a4f-daf96ff0756d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.493306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299201cd-f79f-4120-9a4f-daf96ff0756d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-299201cd-f79f-4120-9a4f-daf96ff0756d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:30.501987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:30.502905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:30.503740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 218/434] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:30.510262 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c5bf7-b337-4267-8eb6-71d47d431631 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:30.537328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c085d5-3a36-4036-92e9-5f83fe07ca67 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.537829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c085d5-3a36-4036-92e9-5f83fe07ca67 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.538116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c085d5-3a36-4036-92e9-5f83fe07ca67 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.540154 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 219/435] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:30.547420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7f6bc-ee14-45ef-920d-6ad654158255 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.547783 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7f6bc-ee14-45ef-920d-6ad654158255 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.548076 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7f6bc-ee14-45ef-920d-6ad654158255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.568547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a8b7f6bc-ee14-45ef-920d-6ad654158255 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:30.570113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a8b7f6bc-ee14-45ef-920d-6ad654158255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:30.620813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:30.643666 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c5bf7-b337-4267-8eb6-71d47d431631 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:30.650336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:30.661963 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 220/436] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:30.683188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17d638-e6f8-45d7-aadd-e2535f0dd687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:30.684172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17d638-e6f8-45d7-aadd-e2535f0dd687 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.684604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17d638-e6f8-45d7-aadd-e2535f0dd687 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.685068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17d638-e6f8-45d7-aadd-e2535f0dd687 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.685998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17d638-e6f8-45d7-aadd-e2535f0dd687 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:30.686518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17d638-e6f8-45d7-aadd-e2535f0dd687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:30.686643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7f6bc-ee14-45ef-920d-6ad654158255 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:30.688247 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17d638-e6f8-45d7-aadd-e2535f0dd687 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:30.702681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 217/437] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:30.708263 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17d638-e6f8-45d7-aadd-e2535f0dd687 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:30.813557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7f6bc-ee14-45ef-920d-6ad654158255 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:30.837437 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17d638-e6f8-45d7-aadd-e2535f0dd687 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:30.849033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46647650-0f73-4706-970b-c8ab701f6b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:30.849847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46647650-0f73-4706-970b-c8ab701f6b72 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.850127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46647650-0f73-4706-970b-c8ab701f6b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.850396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46647650-0f73-4706-970b-c8ab701f6b72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.861547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46647650-0f73-4706-970b-c8ab701f6b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-46647650-0f73-4706-970b-c8ab701f6b72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:30.869586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:30.870662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:30.871715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 221/438] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:30.872534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7270f29-cb20-44a1-9772-c08587a3c95e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.872534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7270f29-cb20-44a1-9772-c08587a3c95e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.872534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7270f29-cb20-44a1-9772-c08587a3c95e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.874121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 222/439] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:30.876276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7270f29-cb20-44a1-9772-c08587a3c95e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:30.878362 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1978369-f9e3-4e07-8e05-c0eea9d19584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:30.879086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1978369-f9e3-4e07-8e05-c0eea9d19584 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.879468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1978369-f9e3-4e07-8e05-c0eea9d19584 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.879848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1978369-f9e3-4e07-8e05-c0eea9d19584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.890965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:30.934752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:30.949068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1978369-f9e3-4e07-8e05-c0eea9d19584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:30.950526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1978369-f9e3-4e07-8e05-c0eea9d19584 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:30.979087 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7270f29-cb20-44a1-9772-c08587a3c95e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.012744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e4e81-8984-4a46-817e-ab354b25eb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.013405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e4e81-8984-4a46-817e-ab354b25eb5c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.013691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e4e81-8984-4a46-817e-ab354b25eb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.013976 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e4e81-8984-4a46-817e-ab354b25eb5c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.015253 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e4e81-8984-4a46-817e-ab354b25eb5c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:31.015569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e4e81-8984-4a46-817e-ab354b25eb5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.016978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e4e81-8984-4a46-817e-ab354b25eb5c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.024769 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 223/440] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:31.025935 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19b8d7-ce65-4958-82ed-d4bf2201cd28 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.026220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19b8d7-ce65-4958-82ed-d4bf2201cd28 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.026495 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19b8d7-ce65-4958-82ed-d4bf2201cd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.042864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6f19b8d7-ce65-4958-82ed-d4bf2201cd28 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:31.044068 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6f19b8d7-ce65-4958-82ed-d4bf2201cd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:31.053611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 218/441] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:31.054443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1978369-f9e3-4e07-8e05-c0eea9d19584 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.094577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.114963 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 224/442] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:31.115640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19b8d7-ce65-4958-82ed-d4bf2201cd28 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.162928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1978369-f9e3-4e07-8e05-c0eea9d19584 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.164191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.196604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0692b-53df-4997-b2d3-236f3a15a8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.197255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0692b-53df-4997-b2d3-236f3a15a8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.197542 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0692b-53df-4997-b2d3-236f3a15a8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.197822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0692b-53df-4997-b2d3-236f3a15a8e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.215134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19b8d7-ce65-4958-82ed-d4bf2201cd28 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.249537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6fa0f-b1a4-4c89-b013-d11eb5bb58e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.250330 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6fa0f-b1a4-4c89-b013-d11eb5bb58e4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.250605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6fa0f-b1a4-4c89-b013-d11eb5bb58e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.250874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6fa0f-b1a4-4c89-b013-d11eb5bb58e4 None admin] PATH_INFO: `/v3/projects/06f70799381a401b8e11df5b00b4192c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.252093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6fa0f-b1a4-4c89-b013-d11eb5bb58e4 None admin] RBAC: Authorizing `identity:get_project(project_id=06f70799381a401b8e11df5b00b4192c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.253524 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6fa0f-b1a4-4c89-b013-d11eb5bb58e4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.255190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 225/443] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/projects/06f70799381a401b8e11df5b00b4192c => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:31.255960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6fa0f-b1a4-4c89-b013-d11eb5bb58e4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.264571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0692b-53df-4997-b2d3-236f3a15a8e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.266509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0692b-53df-4997-b2d3-236f3a15a8e9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.305936 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.351376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6fa0f-b1a4-4c89-b013-d11eb5bb58e4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.364483 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 219/444] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:31.365306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0692b-53df-4997-b2d3-236f3a15a8e9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.370082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.388071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe41ee-e583-4e22-a17d-2a3801d418b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.388864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe41ee-e583-4e22-a17d-2a3801d418b9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.389148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe41ee-e583-4e22-a17d-2a3801d418b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.389423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe41ee-e583-4e22-a17d-2a3801d418b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.399907 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe41ee-e583-4e22-a17d-2a3801d418b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1dbe41ee-e583-4e22-a17d-2a3801d418b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:31.407459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:31.408386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:31.409324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 226/445] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:31.410165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe41ee-e583-4e22-a17d-2a3801d418b9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.466722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0692b-53df-4997-b2d3-236f3a15a8e9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.501400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870f91e-8a2c-406d-a50a-0903fad24560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.502260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870f91e-8a2c-406d-a50a-0903fad24560 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.502547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870f91e-8a2c-406d-a50a-0903fad24560 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.502828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870f91e-8a2c-406d-a50a-0903fad24560 None admin] PATH_INFO: `/v3/groups/6020625330c247569b92315b4f22b7dd` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.504015 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870f91e-8a2c-406d-a50a-0903fad24560 None admin] RBAC: Authorizing `identity:get_group(group_id=6020625330c247569b92315b4f22b7dd)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.505437 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870f91e-8a2c-406d-a50a-0903fad24560 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.507043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 220/446] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/groups/6020625330c247569b92315b4f22b7dd => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:31.507358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe41ee-e583-4e22-a17d-2a3801d418b9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.507829 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870f91e-8a2c-406d-a50a-0903fad24560 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.522912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.541024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ca33c-10b3-4b51-a280-b45f2c536cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.541874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ca33c-10b3-4b51-a280-b45f2c536cac None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.542162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ca33c-10b3-4b51-a280-b45f2c536cac None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.542439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ca33c-10b3-4b51-a280-b45f2c536cac None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.543617 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ca33c-10b3-4b51-a280-b45f2c536cac None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.545094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ca33c-10b3-4b51-a280-b45f2c536cac None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.546797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 227/447] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:31.547584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ca33c-10b3-4b51-a280-b45f2c536cac None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.592892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.616080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870f91e-8a2c-406d-a50a-0903fad24560 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.647673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ca33c-10b3-4b51-a280-b45f2c536cac None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.648634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4937a88-3171-4d07-ac66-bef2418418cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.649289 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4937a88-3171-4d07-ac66-bef2418418cf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.649564 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4937a88-3171-4d07-ac66-bef2418418cf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.649844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4937a88-3171-4d07-ac66-bef2418418cf None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.650576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4937a88-3171-4d07-ac66-bef2418418cf None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:31.650878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4937a88-3171-4d07-ac66-bef2418418cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.652280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4937a88-3171-4d07-ac66-bef2418418cf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.660610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 221/448] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:31.661463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4937a88-3171-4d07-ac66-bef2418418cf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.681146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d6fb7-16fd-4777-ae0e-5a302c8c2e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.681803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d6fb7-16fd-4777-ae0e-5a302c8c2e2b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.682092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d6fb7-16fd-4777-ae0e-5a302c8c2e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.682374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d6fb7-16fd-4777-ae0e-5a302c8c2e2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.683211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d6fb7-16fd-4777-ae0e-5a302c8c2e2b None admin] RBAC: Adding query filter params (group.id=6020625330c247569b92315b4f22b7dd, role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=06f70799381a401b8e11df5b00b4192c) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:31.683529 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d6fb7-16fd-4777-ae0e-5a302c8c2e2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.685002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d6fb7-16fd-4777-ae0e-5a302c8c2e2b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.691272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 228/449] 10.222.0.65 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/role_assignments?group.id=6020625330c247569b92315b4f22b7dd&scope.project.id=06f70799381a401b8e11df5b00b4192c&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:31.692066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d6fb7-16fd-4777-ae0e-5a302c8c2e2b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.747339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.784324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4937a88-3171-4d07-ac66-bef2418418cf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.800935 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d6fb7-16fd-4777-ae0e-5a302c8c2e2b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.810550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.828570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9102d7-6ae0-4f47-bd4b-8917cc7314f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.829570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9102d7-6ae0-4f47-bd4b-8917cc7314f1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.830058 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9102d7-6ae0-4f47-bd4b-8917cc7314f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.830343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9102d7-6ae0-4f47-bd4b-8917cc7314f1 None admin] PATH_INFO: `/v3/groups/5c4af3860a484ddda332a48e2765d284` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.832198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9102d7-6ae0-4f47-bd4b-8917cc7314f1 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4af3860a484ddda332a48e2765d284)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.834389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9102d7-6ae0-4f47-bd4b-8917cc7314f1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.836106 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 222/450] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/groups/5c4af3860a484ddda332a48e2765d284 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:31.841954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9102d7-6ae0-4f47-bd4b-8917cc7314f1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.842892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1ddc7-3b5d-40d0-8e3d-095cdc3e183a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.843563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1ddc7-3b5d-40d0-8e3d-095cdc3e183a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.843887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1ddc7-3b5d-40d0-8e3d-095cdc3e183a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.844172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1ddc7-3b5d-40d0-8e3d-095cdc3e183a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.922349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1ddc7-3b5d-40d0-8e3d-095cdc3e183a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.923854 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1ddc7-3b5d-40d0-8e3d-095cdc3e183a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.952670 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9102d7-6ae0-4f47-bd4b-8917cc7314f1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.985846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36266796-bdb2-45ca-b958-78e6b403b04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.986679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36266796-bdb2-45ca-b958-78e6b403b04d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.986954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36266796-bdb2-45ca-b958-78e6b403b04d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.987226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36266796-bdb2-45ca-b958-78e6b403b04d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.988082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36266796-bdb2-45ca-b958-78e6b403b04d None admin] RBAC: Adding query filter params (group.id=5c4af3860a484ddda332a48e2765d284, role.id=8a0cfaebc67a497a98459da3f1a82ef9, scope.project.id=73330eaa9cf649e283e878927d1e2da5) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:31.988392 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36266796-bdb2-45ca-b958-78e6b403b04d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.989800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36266796-bdb2-45ca-b958-78e6b403b04d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.995424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 223/451] 10.222.0.65 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/role_assignments?group.id=5c4af3860a484ddda332a48e2765d284&scope.project.id=73330eaa9cf649e283e878927d1e2da5&role.id=8a0cfaebc67a497a98459da3f1a82ef9 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:32.023251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 229/452] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:32.029076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36266796-bdb2-45ca-b958-78e6b403b04d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:32.032573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:32.134248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36266796-bdb2-45ca-b958-78e6b403b04d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:32.167515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaedcbe-95cb-42ba-b998-f6af5dbc90c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:32.168411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaedcbe-95cb-42ba-b998-f6af5dbc90c5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:32.168690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaedcbe-95cb-42ba-b998-f6af5dbc90c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:32.168961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaedcbe-95cb-42ba-b998-f6af5dbc90c5 None admin] PATH_INFO: `/v3/projects/73330eaa9cf649e283e878927d1e2da5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:32.170127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaedcbe-95cb-42ba-b998-f6af5dbc90c5 None admin] RBAC: Authorizing `identity:get_project(project_id=73330eaa9cf649e283e878927d1e2da5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:32.171570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaedcbe-95cb-42ba-b998-f6af5dbc90c5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:32.173281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 224/453] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:32 2022] GET /identity/v3/projects/73330eaa9cf649e283e878927d1e2da5 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:32.178891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1ddc7-3b5d-40d0-8e3d-095cdc3e183a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:32.189978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:32.276264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1ddc7-3b5d-40d0-8e3d-095cdc3e183a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:32.308566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf5d91-c219-46f4-80c6-b7cf828c6233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:32.311099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf5d91-c219-46f4-80c6-b7cf828c6233 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:32.311099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf5d91-c219-46f4-80c6-b7cf828c6233 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:32.311099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf5d91-c219-46f4-80c6-b7cf828c6233 None admin] PATH_INFO: `/v3/groups/5c4af3860a484ddda332a48e2765d284` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:32.311871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf5d91-c219-46f4-80c6-b7cf828c6233 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4af3860a484ddda332a48e2765d284)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:32.314013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf5d91-c219-46f4-80c6-b7cf828c6233 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:32.314860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 230/454] 10.222.0.65 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:32 2022] GET /identity/v3/groups/5c4af3860a484ddda332a48e2765d284 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:32.320807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaedcbe-95cb-42ba-b998-f6af5dbc90c5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:32.387357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:32.416829 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaedcbe-95cb-42ba-b998-f6af5dbc90c5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:32.448798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7e3c5-3ecf-4d0e-9d41-8c207b397e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:32.449430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7e3c5-3ecf-4d0e-9d41-8c207b397e80 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:32.449711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7e3c5-3ecf-4d0e-9d41-8c207b397e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:32.449986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7e3c5-3ecf-4d0e-9d41-8c207b397e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:32.450799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7e3c5-3ecf-4d0e-9d41-8c207b397e80 None admin] RBAC: Adding query filter params (group.id=5c4af3860a484ddda332a48e2765d284, role.id=7d15e336f8034a1fbdef48cfb7c4f717, scope.project.id=73330eaa9cf649e283e878927d1e2da5) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:32.451109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7e3c5-3ecf-4d0e-9d41-8c207b397e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:32.452570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7e3c5-3ecf-4d0e-9d41-8c207b397e80 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:32.458108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 225/455] 10.222.0.65 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:32 2022] GET /identity/v3/role_assignments?group.id=5c4af3860a484ddda332a48e2765d284&scope.project.id=73330eaa9cf649e283e878927d1e2da5&role.id=7d15e336f8034a1fbdef48cfb7c4f717 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:35.144214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f906609-fa01-474b-8c13-0cd6eafc20d9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.144715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f906609-fa01-474b-8c13-0cd6eafc20d9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.144789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f906609-fa01-474b-8c13-0cd6eafc20d9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.146390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 231/456] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:35.151212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb1711-5ad4-4136-a4ce-1bfa92955e39 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.152032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb1711-5ad4-4136-a4ce-1bfa92955e39 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.152549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb1711-5ad4-4136-a4ce-1bfa92955e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.180710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6dcb1711-5ad4-4136-a4ce-1bfa92955e39 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:35.182233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6dcb1711-5ad4-4136-a4ce-1bfa92955e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:35.273413 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 226/457] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:35.320493 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f906609-fa01-474b-8c13-0cd6eafc20d9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:35.343309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cda52-f386-4bb5-9c65-087ea9809ba8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.343615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cda52-f386-4bb5-9c65-087ea9809ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.343906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cda52-f386-4bb5-9c65-087ea9809ba8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.345554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 227/458] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:35.350129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7645c-56b0-4462-b416-7d28f83efa68 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.350338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7645c-56b0-4462-b416-7d28f83efa68 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.350620 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7645c-56b0-4462-b416-7d28f83efa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.378029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b2c7645c-56b0-4462-b416-7d28f83efa68 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:35.379202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b2c7645c-56b0-4462-b416-7d28f83efa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:35.455341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:35.492386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:35.493286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f906609-fa01-474b-8c13-0cd6eafc20d9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:35.510393 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 228/459] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:35.551583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595de26e-1965-4416-906f-086fcb3340f2 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.551887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595de26e-1965-4416-906f-086fcb3340f2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.552166 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595de26e-1965-4416-906f-086fcb3340f2 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.553680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 229/460] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:35.554398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a7736-3a11-4cc4-be58-2946ed4d3812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:35.555195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a7736-3a11-4cc4-be58-2946ed4d3812 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.555472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a7736-3a11-4cc4-be58-2946ed4d3812 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.555771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a7736-3a11-4cc4-be58-2946ed4d3812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.566082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694fcb2-f8c3-436a-a0d7-68b6ad10e4b0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.566082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694fcb2-f8c3-436a-a0d7-68b6ad10e4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.566082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694fcb2-f8c3-436a-a0d7-68b6ad10e4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.569035 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a7736-3a11-4cc4-be58-2946ed4d3812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:35.570744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a7736-3a11-4cc4-be58-2946ed4d3812 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:35.583077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d694fcb2-f8c3-436a-a0d7-68b6ad10e4b0 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:35.584376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d694fcb2-f8c3-436a-a0d7-68b6ad10e4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e72a7736-3a11-4cc4-be58-2946ed4d3812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:35.591679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:35.592719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:35.593722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:35.593722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:35.593722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:35.593722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:35.595703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 232/461] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:35.596787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a7736-3a11-4cc4-be58-2946ed4d3812 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:35.663753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 230/462] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:35.663753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694fcb2-f8c3-436a-a0d7-68b6ad10e4b0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:35.728694 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a7736-3a11-4cc4-be58-2946ed4d3812 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:35.757557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:35.764353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694fcb2-f8c3-436a-a0d7-68b6ad10e4b0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:35.766859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b49ce2-ce13-4c75-afef-40628b6fed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:35.767840 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b49ce2-ce13-4c75-afef-40628b6fed96 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.768263 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b49ce2-ce13-4c75-afef-40628b6fed96 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.768709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b49ce2-ce13-4c75-afef-40628b6fed96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.780940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b49ce2-ce13-4c75-afef-40628b6fed96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:35.782727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b49ce2-ce13-4c75-afef-40628b6fed96 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:35.789440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:35.801369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c443dd2-b68d-4d51-bf8c-aa57a0de483e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:35.802037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c443dd2-b68d-4d51-bf8c-aa57a0de483e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.802321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c443dd2-b68d-4d51-bf8c-aa57a0de483e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.802600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c443dd2-b68d-4d51-bf8c-aa57a0de483e None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.803333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c443dd2-b68d-4d51-bf8c-aa57a0de483e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:35.803638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c443dd2-b68d-4d51-bf8c-aa57a0de483e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e6b49ce2-ce13-4c75-afef-40628b6fed96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:35.804242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:35.805257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:35.806300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:35.806300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:35.806300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:35.806300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:35.806517 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c443dd2-b68d-4d51-bf8c-aa57a0de483e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:35.812751 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 233/463] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:35.813820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b49ce2-ce13-4c75-afef-40628b6fed96 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:35.816913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 231/464] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:35.817768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c443dd2-b68d-4d51-bf8c-aa57a0de483e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:35.922843 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b49ce2-ce13-4c75-afef-40628b6fed96 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:35.924722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c443dd2-b68d-4d51-bf8c-aa57a0de483e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:35.960023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b60011-a956-4b13-8564-154c3ba50e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:35.960619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafdc0e-43eb-4d34-b70c-8c971aacc46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:35.960825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b60011-a956-4b13-8564-154c3ba50e51 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.961089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b60011-a956-4b13-8564-154c3ba50e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.961282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafdc0e-43eb-4d34-b70c-8c971aacc46c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.961376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b60011-a956-4b13-8564-154c3ba50e51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.961560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafdc0e-43eb-4d34-b70c-8c971aacc46c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.961887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafdc0e-43eb-4d34-b70c-8c971aacc46c None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.962543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aafdc0e-43eb-4d34-b70c-8c971aacc46c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:35.962848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aafdc0e-43eb-4d34-b70c-8c971aacc46c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:35.964240 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aafdc0e-43eb-4d34-b70c-8c971aacc46c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:35.972874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b60011-a956-4b13-8564-154c3ba50e51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:35.974314 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b60011-a956-4b13-8564-154c3ba50e51 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:35.976598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 232/465] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:35.977376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafdc0e-43eb-4d34-b70c-8c971aacc46c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-21b60011-a956-4b13-8564-154c3ba50e51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:35.995932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:35.996845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:35.997690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:35.997690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:35.997690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:35.997690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:35.997690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 234/466] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:35.998648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb731ed-ea9d-42d0-bead-a8bc4b9bbe1b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.998937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb731ed-ea9d-42d0-bead-a8bc4b9bbe1b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.999251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb731ed-ea9d-42d0-bead-a8bc4b9bbe1b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.000658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 235/467] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:36.004437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb731ed-ea9d-42d0-bead-a8bc4b9bbe1b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.029552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.044070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.091529 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafdc0e-43eb-4d34-b70c-8c971aacc46c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.128989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb731ed-ea9d-42d0-bead-a8bc4b9bbe1b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.130073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dd823-faa3-4b9d-9349-e3202d6c146c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.130786 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dd823-faa3-4b9d-9349-e3202d6c146c None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.131070 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dd823-faa3-4b9d-9349-e3202d6c146c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.131349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dd823-faa3-4b9d-9349-e3202d6c146c None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.132080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7dd823-faa3-4b9d-9349-e3202d6c146c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.133522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7dd823-faa3-4b9d-9349-e3202d6c146c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.163849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 233/468] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:35 2022] POST /identity/v3/users => generated 246 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:36.165539 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dd823-faa3-4b9d-9349-e3202d6c146c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.171703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29fd43-6f97-4064-bfe9-1280f81f216e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.171703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29fd43-6f97-4064-bfe9-1280f81f216e None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.171703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29fd43-6f97-4064-bfe9-1280f81f216e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.171703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29fd43-6f97-4064-bfe9-1280f81f216e None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.172436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29fd43-6f97-4064-bfe9-1280f81f216e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.172436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29fd43-6f97-4064-bfe9-1280f81f216e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.200466 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 236/469] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:36 2022] POST /identity/v3/users => generated 248 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:36.202764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc220e6-f250-429d-ad0c-43f2a88450c8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.203104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc220e6-f250-429d-ad0c-43f2a88450c8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.203415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc220e6-f250-429d-ad0c-43f2a88450c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.241116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7bc220e6-f250-429d-ad0c-43f2a88450c8 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:36.242433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7bc220e6-f250-429d-ad0c-43f2a88450c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:36.293323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.297505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dd823-faa3-4b9d-9349-e3202d6c146c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.353675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 237/470] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:36.360911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4d95e-665d-40d9-8fb7-2eff0d521812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.361841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4d95e-665d-40d9-8fb7-2eff0d521812 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.362291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4d95e-665d-40d9-8fb7-2eff0d521812 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.362727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4d95e-665d-40d9-8fb7-2eff0d521812 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.363599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4d95e-665d-40d9-8fb7-2eff0d521812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:36.364102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4d95e-665d-40d9-8fb7-2eff0d521812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.365937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4d95e-665d-40d9-8fb7-2eff0d521812 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.379589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 234/471] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:36.379934 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc220e6-f250-429d-ad0c-43f2a88450c8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.392273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4d95e-665d-40d9-8fb7-2eff0d521812 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.406464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.522403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc220e6-f250-429d-ad0c-43f2a88450c8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.538131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4d95e-665d-40d9-8fb7-2eff0d521812 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.561916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fee5a-408b-499a-aa2b-0c0c0a43e114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.564344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fee5a-408b-499a-aa2b-0c0c0a43e114 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.564841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fee5a-408b-499a-aa2b-0c0c0a43e114 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.565308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fee5a-408b-499a-aa2b-0c0c0a43e114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.578359 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749fee5a-408b-499a-aa2b-0c0c0a43e114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.579788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd955eeb-2e05-47fb-af88-0ece1d923ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.580282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749fee5a-408b-499a-aa2b-0c0c0a43e114 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.580468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd955eeb-2e05-47fb-af88-0ece1d923ccc None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.580751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd955eeb-2e05-47fb-af88-0ece1d923ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.581025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd955eeb-2e05-47fb-af88-0ece1d923ccc None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.581738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd955eeb-2e05-47fb-af88-0ece1d923ccc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.583356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd955eeb-2e05-47fb-af88-0ece1d923ccc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-749fee5a-408b-499a-aa2b-0c0c0a43e114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:36.601484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:36.602395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:36.603234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:36.603234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:36.603234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:36.603234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:36.603464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 238/472] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:36.604120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 235/473] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:36 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:49:36.604764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246093-43b2-4015-a2b8-253e77022f1a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.605064 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246093-43b2-4015-a2b8-253e77022f1a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.605376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246093-43b2-4015-a2b8-253e77022f1a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.606960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 239/474] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:36.610242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd955eeb-2e05-47fb-af88-0ece1d923ccc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.612730 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5351c-de6a-4696-abb9-5f5cd56f87d8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.613028 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5351c-de6a-4696-abb9-5f5cd56f87d8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.613312 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5351c-de6a-4696-abb9-5f5cd56f87d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.639094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dde5351c-de6a-4696-abb9-5f5cd56f87d8 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:36.640278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.640891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dde5351c-de6a-4696-abb9-5f5cd56f87d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:36.728224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.738571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 240/475] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:36.740712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd955eeb-2e05-47fb-af88-0ece1d923ccc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.774463 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5351c-de6a-4696-abb9-5f5cd56f87d8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.779755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853adf5-8aa7-4abd-ac39-963e2f1a7f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.781833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853adf5-8aa7-4abd-ac39-963e2f1a7f0f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.782354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853adf5-8aa7-4abd-ac39-963e2f1a7f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.782830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853adf5-8aa7-4abd-ac39-963e2f1a7f0f None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.783830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853adf5-8aa7-4abd-ac39-963e2f1a7f0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:36.784342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853adf5-8aa7-4abd-ac39-963e2f1a7f0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.785916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853adf5-8aa7-4abd-ac39-963e2f1a7f0f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.800518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 236/476] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:36.805567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853adf5-8aa7-4abd-ac39-963e2f1a7f0f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.917715 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.928103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5351c-de6a-4696-abb9-5f5cd56f87d8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.941096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853adf5-8aa7-4abd-ac39-963e2f1a7f0f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.969197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5702b-559e-478d-83b1-68ee91c7121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.970190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5702b-559e-478d-83b1-68ee91c7121a None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.970690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5702b-559e-478d-83b1-68ee91c7121a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.971163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5702b-559e-478d-83b1-68ee91c7121a None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.972181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5702b-559e-478d-83b1-68ee91c7121a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.973748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5702b-559e-478d-83b1-68ee91c7121a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.979638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b1f6c-9102-4589-b7ed-3093e0060de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.979638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b1f6c-9102-4589-b7ed-3093e0060de0 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.980051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b1f6c-9102-4589-b7ed-3093e0060de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.980373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b1f6c-9102-4589-b7ed-3093e0060de0 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.981213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b1f6c-9102-4589-b7ed-3093e0060de0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.982262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.sql.core [None req-19b5702b-559e-478d-83b1-68ee91c7121a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 10:49:36.982262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 10:49:36.982262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [parameters: {'id': '431debb327bd42bf99294a5f86e8691a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 10:49:36.982262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80206) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 10:49:36.982637 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b1f6c-9102-4589-b7ed-3093e0060de0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-19b5702b-559e-478d-83b1-68ee91c7121a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jun 23 10:49:37.003096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return ref.to_dict() Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 10:49:37.004151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application yield resource Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise self.value Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application session.commit() Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 10:49:37.005069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.flush() Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application compat.raise_( Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 10:49:37.005929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 10:49:37.009369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 10:49:37.011499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [parameters: {'id': '431debb327bd42bf99294a5f86e8691a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:37.013581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 10:49:37.015714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:37.021303 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 241/477] 10.222.0.65 () {60 vars in 1131 bytes} [Thu Jun 23 10:49:36 2022] POST /identity/v3/roles => generated 144 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:37.021452 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 237/478] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:36 2022] POST /identity/v3/users => generated 249 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:37.024034 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5702b-559e-478d-83b1-68ee91c7121a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:37.105857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:37.149195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5702b-559e-478d-83b1-68ee91c7121a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:37.192352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f6c94-1db1-4a13-a7be-af14ecc1b19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:37.193434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f6c94-1db1-4a13-a7be-af14ecc1b19e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:37.193717 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f6c94-1db1-4a13-a7be-af14ecc1b19e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:37.193993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f6c94-1db1-4a13-a7be-af14ecc1b19e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:37.205164 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f6c94-1db1-4a13-a7be-af14ecc1b19e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-345f6c94-1db1-4a13-a7be-af14ecc1b19e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:37.219864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:37.220775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:37.221586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:37.221586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:37.221586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:37.221586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:37.221586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:37.221586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:37.221586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:37.221586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:37.221586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:37.221586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:37.221586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:37.221586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:37.222328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 242/479] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:37.228203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b1f6c-9102-4589-b7ed-3093e0060de0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:37.349383 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b1f6c-9102-4589-b7ed-3093e0060de0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:37.380515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:37.385646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee073f8f-91e4-4d62-9dad-d5bffaf3c1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:37.386566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee073f8f-91e4-4d62-9dad-d5bffaf3c1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:37.387366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee073f8f-91e4-4d62-9dad-d5bffaf3c1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:37.387595 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee073f8f-91e4-4d62-9dad-d5bffaf3c1a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:37.388532 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee073f8f-91e4-4d62-9dad-d5bffaf3c1a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:37.389060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee073f8f-91e4-4d62-9dad-d5bffaf3c1a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:37.390752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee073f8f-91e4-4d62-9dad-d5bffaf3c1a0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:37.399757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 238/480] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:38.684398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876a9c3-c662-4105-8793-51e126ee6b66 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:38.685397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876a9c3-c662-4105-8793-51e126ee6b66 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:38.685871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876a9c3-c662-4105-8793-51e126ee6b66 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:38.688257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 243/481] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:38.693036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b683793-d09e-4be3-bdef-140998e10153 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:38.693815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b683793-d09e-4be3-bdef-140998e10153 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:38.694309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b683793-d09e-4be3-bdef-140998e10153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:38.716525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3b683793-d09e-4be3-bdef-140998e10153 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:38.718010 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3b683793-d09e-4be3-bdef-140998e10153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:38.807318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 239/482] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:38.842199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876a9c3-c662-4105-8793-51e126ee6b66 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:38.981354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876a9c3-c662-4105-8793-51e126ee6b66 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.015028 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:39.029158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252223b-429f-4aa4-8e54-651a4b31ba31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.030883 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252223b-429f-4aa4-8e54-651a4b31ba31 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.031598 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252223b-429f-4aa4-8e54-651a4b31ba31 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.031905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252223b-429f-4aa4-8e54-651a4b31ba31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.046052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252223b-429f-4aa4-8e54-651a4b31ba31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5252223b-429f-4aa4-8e54-651a4b31ba31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:39.053652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:39.054526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:39.055302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 244/483] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:39.060580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b683793-d09e-4be3-bdef-140998e10153 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.120225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f863092-66cf-4df9-b077-32cc17c2aab4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.120514 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f863092-66cf-4df9-b077-32cc17c2aab4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.120780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f863092-66cf-4df9-b077-32cc17c2aab4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.122574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 245/484] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:39.128040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c88b71-660b-4f10-9ca0-44682d702044 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.128326 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c88b71-660b-4f10-9ca0-44682d702044 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.128603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c88b71-660b-4f10-9ca0-44682d702044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.157838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d0c88b71-660b-4f10-9ca0-44682d702044 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:39.159029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d0c88b71-660b-4f10-9ca0-44682d702044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:39.258822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 246/485] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:39.276902 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b683793-d09e-4be3-bdef-140998e10153 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.294516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c88b71-660b-4f10-9ca0-44682d702044 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.304915 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:39.316602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2a1c7-1a64-4cb7-b9f3-344e704edd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.317285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2a1c7-1a64-4cb7-b9f3-344e704edd48 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.317573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2a1c7-1a64-4cb7-b9f3-344e704edd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.317856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2a1c7-1a64-4cb7-b9f3-344e704edd48 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.318617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2a1c7-1a64-4cb7-b9f3-344e704edd48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:39.318932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2a1c7-1a64-4cb7-b9f3-344e704edd48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:39.320339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2a1c7-1a64-4cb7-b9f3-344e704edd48 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:39.340872 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 240/486] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:39.347112 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2a1c7-1a64-4cb7-b9f3-344e704edd48 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.483937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2a1c7-1a64-4cb7-b9f3-344e704edd48 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.519869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a2533-9d5e-40af-a791-28c33389659c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.520682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2533-9d5e-40af-a791-28c33389659c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.520969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2533-9d5e-40af-a791-28c33389659c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.521255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2533-9d5e-40af-a791-28c33389659c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.528775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c88b71-660b-4f10-9ca0-44682d702044 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.533023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a2533-9d5e-40af-a791-28c33389659c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:39.534486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a2533-9d5e-40af-a791-28c33389659c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:39.544352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9f8a2533-9d5e-40af-a791-28c33389659c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:39.553749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:39.554621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:39.555484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:39.555484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:39.555484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:39.555484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:39.555484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 241/487] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:39.556528 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a048d1-cba6-4f07-b285-fa5c6c6cf6fd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.556811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a048d1-cba6-4f07-b285-fa5c6c6cf6fd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.557086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a048d1-cba6-4f07-b285-fa5c6c6cf6fd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.558459 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 242/488] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:39.559406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e3c6b-a7f7-4e1d-875f-baef455206e5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.559717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e3c6b-a7f7-4e1d-875f-baef455206e5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.559987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e3c6b-a7f7-4e1d-875f-baef455206e5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.561847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 243/489] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:39.566022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e3c6b-a7f7-4e1d-875f-baef455206e5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.575636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579f9ea-c60f-4bc9-8655-579dc82610f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.577309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579f9ea-c60f-4bc9-8655-579dc82610f6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.577595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579f9ea-c60f-4bc9-8655-579dc82610f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.577873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579f9ea-c60f-4bc9-8655-579dc82610f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.591216 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579f9ea-c60f-4bc9-8655-579dc82610f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5579f9ea-c60f-4bc9-8655-579dc82610f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:39.601352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:39.602290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:39.603649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 247/490] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:39.607162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ea3e7-091c-4fc0-b92b-283cf5de1d60 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.607162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ea3e7-091c-4fc0-b92b-283cf5de1d60 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.607162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ea3e7-091c-4fc0-b92b-283cf5de1d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.607799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:39.625797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1b4ea3e7-091c-4fc0-b92b-283cf5de1d60 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:39.627060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1b4ea3e7-091c-4fc0-b92b-283cf5de1d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:39.670629 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e3c6b-a7f7-4e1d-875f-baef455206e5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.695830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 248/491] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:39.697113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137daa9e-235c-4c51-9edf-ea387e200bf5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.697399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137daa9e-235c-4c51-9edf-ea387e200bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.697676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137daa9e-235c-4c51-9edf-ea387e200bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.710374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8287e4-f1dd-4f43-8ad3-db9906322829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.711275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8287e4-f1dd-4f43-8ad3-db9906322829 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.711683 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8287e4-f1dd-4f43-8ad3-db9906322829 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.712068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8287e4-f1dd-4f43-8ad3-db9906322829 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.712948 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8287e4-f1dd-4f43-8ad3-db9906322829 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:39.713364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8287e4-f1dd-4f43-8ad3-db9906322829 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:39.714913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8287e4-f1dd-4f43-8ad3-db9906322829 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:39.718268 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-137daa9e-235c-4c51-9edf-ea387e200bf5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:39.719628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-137daa9e-235c-4c51-9edf-ea387e200bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:39.727949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 244/492] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:39.728891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8287e4-f1dd-4f43-8ad3-db9906322829 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.793799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 249/493] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:39.794890 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137daa9e-235c-4c51-9edf-ea387e200bf5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.834930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:39.838411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:39.843885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8287e4-f1dd-4f43-8ad3-db9906322829 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.889056 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f7f5c-5d51-4f20-9494-b896ccc5ab38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.890489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f7f5c-5d51-4f20-9494-b896ccc5ab38 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.890783 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f7f5c-5d51-4f20-9494-b896ccc5ab38 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.891064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f7f5c-5d51-4f20-9494-b896ccc5ab38 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.892223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f7f5c-5d51-4f20-9494-b896ccc5ab38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:39.892542 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f7f5c-5d51-4f20-9494-b896ccc5ab38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:39.894438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f7f5c-5d51-4f20-9494-b896ccc5ab38 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:39.903278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 245/494] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:39.907766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137daa9e-235c-4c51-9edf-ea387e200bf5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.908924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f7f5c-5d51-4f20-9494-b896ccc5ab38 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.944673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3296b87-d6c6-4b48-83fa-e5f92dfcc616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.945470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3296b87-d6c6-4b48-83fa-e5f92dfcc616 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.945755 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3296b87-d6c6-4b48-83fa-e5f92dfcc616 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.946032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3296b87-d6c6-4b48-83fa-e5f92dfcc616 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.956157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3296b87-d6c6-4b48-83fa-e5f92dfcc616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c3296b87-d6c6-4b48-83fa-e5f92dfcc616 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:39.963913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:39.964785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:39.965572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 250/495] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:39.966234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3296b87-d6c6-4b48-83fa-e5f92dfcc616 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.011454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f7f5c-5d51-4f20-9494-b896ccc5ab38 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.043638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a4af0-cf84-46cd-b467-f601167d3267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.044496 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a4af0-cf84-46cd-b467-f601167d3267 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.044730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a4af0-cf84-46cd-b467-f601167d3267 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.044999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a4af0-cf84-46cd-b467-f601167d3267 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.053016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a4af0-cf84-46cd-b467-f601167d3267 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.054441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a4af0-cf84-46cd-b467-f601167d3267 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.057622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.065738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3296b87-d6c6-4b48-83fa-e5f92dfcc616 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f99a4af0-cf84-46cd-b467-f601167d3267 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:40.070011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:40.070888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:40.071799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:40.071799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:40.071799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:40.071799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:40.071799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:40.071799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:40.071799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:40.071799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:40.071799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:49:40.071799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:40.071799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 246/496] 10.222.0.65 () {56 vars in 1093 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:40.072352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a4af0-cf84-46cd-b467-f601167d3267 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.093788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.108491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0ea71-ba8f-48b6-9869-dd6cd431e548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.109291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0ea71-ba8f-48b6-9869-dd6cd431e548 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.109571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0ea71-ba8f-48b6-9869-dd6cd431e548 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.109840 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0ea71-ba8f-48b6-9869-dd6cd431e548 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.120421 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0ea71-ba8f-48b6-9869-dd6cd431e548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-52f0ea71-ba8f-48b6-9869-dd6cd431e548 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:40.128561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:40.129468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:40.130291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 251/497] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:40.131212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e93baf-a8a2-4e06-8ebe-4c7c2348f5ed None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.131212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e93baf-a8a2-4e06-8ebe-4c7c2348f5ed None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.131447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e93baf-a8a2-4e06-8ebe-4c7c2348f5ed None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.132704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 252/498] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:40.133384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e93baf-a8a2-4e06-8ebe-4c7c2348f5ed None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.177193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a4af0-cf84-46cd-b467-f601167d3267 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.210391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908148a8-778e-431f-b84b-5c23eceaed87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.211178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908148a8-778e-431f-b84b-5c23eceaed87 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.211457 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908148a8-778e-431f-b84b-5c23eceaed87 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.211763 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908148a8-778e-431f-b84b-5c23eceaed87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.221982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908148a8-778e-431f-b84b-5c23eceaed87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.223456 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908148a8-778e-431f-b84b-5c23eceaed87 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.234574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e93baf-a8a2-4e06-8ebe-4c7c2348f5ed None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-908148a8-778e-431f-b84b-5c23eceaed87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:40.240522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:40.241513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:40.242593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:40.242593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:40.242593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:40.242593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:40.242593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 247/499] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:40.242593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908148a8-778e-431f-b84b-5c23eceaed87 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.269775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1d4cb-c331-440e-8b4f-ce5c6347791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.270441 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1d4cb-c331-440e-8b4f-ce5c6347791d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.270725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1d4cb-c331-440e-8b4f-ce5c6347791d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.271004 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1d4cb-c331-440e-8b4f-ce5c6347791d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.271785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1d4cb-c331-440e-8b4f-ce5c6347791d None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:40.272094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1d4cb-c331-440e-8b4f-ce5c6347791d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.273460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1d4cb-c331-440e-8b4f-ce5c6347791d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.281640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 253/500] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:40.282412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1d4cb-c331-440e-8b4f-ce5c6347791d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.300648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.310215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.338778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908148a8-778e-431f-b84b-5c23eceaed87 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.372315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb14422-ab2a-469d-accd-58c9494719f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.372977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb14422-ab2a-469d-accd-58c9494719f1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.373257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb14422-ab2a-469d-accd-58c9494719f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.373570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb14422-ab2a-469d-accd-58c9494719f1 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.374379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb14422-ab2a-469d-accd-58c9494719f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:40.374686 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb14422-ab2a-469d-accd-58c9494719f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.376127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb14422-ab2a-469d-accd-58c9494719f1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.381633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1d4cb-c331-440e-8b4f-ce5c6347791d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.411558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 248/501] 10.222.0.65 () {56 vars in 1133 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:40.412897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eb58d-2af1-4e8f-b874-59bbcaa85fd2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.413187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eb58d-2af1-4e8f-b874-59bbcaa85fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.413461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eb58d-2af1-4e8f-b874-59bbcaa85fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.417476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfd493-cdd1-42f2-8a25-e6f9ad69b6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.418151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfd493-cdd1-42f2-8a25-e6f9ad69b6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.418439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfd493-cdd1-42f2-8a25-e6f9ad69b6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.418721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfd493-cdd1-42f2-8a25-e6f9ad69b6c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.419471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfd493-cdd1-42f2-8a25-e6f9ad69b6c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:40.419807 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfd493-cdd1-42f2-8a25-e6f9ad69b6c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.421798 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfd493-cdd1-42f2-8a25-e6f9ad69b6c1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.429873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 254/502] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:40.430656 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfd493-cdd1-42f2-8a25-e6f9ad69b6c1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.430982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9c3eb58d-2af1-4e8f-b874-59bbcaa85fd2 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:40.432216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9c3eb58d-2af1-4e8f-b874-59bbcaa85fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:40.496672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 249/503] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:40.497483 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3eb58d-2af1-4e8f-b874-59bbcaa85fd2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.509904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.533276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfd493-cdd1-42f2-8a25-e6f9ad69b6c1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.557022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.566963 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b19eb-3922-4a30-9ff5-c357a1d4f149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.567622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b19eb-3922-4a30-9ff5-c357a1d4f149 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.567941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b19eb-3922-4a30-9ff5-c357a1d4f149 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.568215 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b19eb-3922-4a30-9ff5-c357a1d4f149 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.568930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b19eb-3922-4a30-9ff5-c357a1d4f149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:40.569235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b19eb-3922-4a30-9ff5-c357a1d4f149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.570598 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b19eb-3922-4a30-9ff5-c357a1d4f149 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.587278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 255/504] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:40.588129 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b19eb-3922-4a30-9ff5-c357a1d4f149 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.601368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3eb58d-2af1-4e8f-b874-59bbcaa85fd2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.635373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9846fc4-97da-496b-bdd9-9418aa00912f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.636181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9846fc4-97da-496b-bdd9-9418aa00912f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.636460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9846fc4-97da-496b-bdd9-9418aa00912f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.636739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9846fc4-97da-496b-bdd9-9418aa00912f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.647004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9846fc4-97da-496b-bdd9-9418aa00912f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.648600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9846fc4-97da-496b-bdd9-9418aa00912f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d9846fc4-97da-496b-bdd9-9418aa00912f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:40.666510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:40.667360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:40.668511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:40.668511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:40.668511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:40.668511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:40.668511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 250/505] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:40.668511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9846fc4-97da-496b-bdd9-9418aa00912f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.692448 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b19eb-3922-4a30-9ff5-c357a1d4f149 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.725461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fc23d-b4c0-42cf-bc03-55fc6185ad89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.726112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fc23d-b4c0-42cf-bc03-55fc6185ad89 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.726376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fc23d-b4c0-42cf-bc03-55fc6185ad89 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.726647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fc23d-b4c0-42cf-bc03-55fc6185ad89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.734822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.765627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9846fc4-97da-496b-bdd9-9418aa00912f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.777910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.794317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fc23d-b4c0-42cf-bc03-55fc6185ad89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.795842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fc23d-b4c0-42cf-bc03-55fc6185ad89 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.799228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba572a83-9e94-4dfd-8a9f-2d9d5ad61a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.800042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba572a83-9e94-4dfd-8a9f-2d9d5ad61a8d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.800327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba572a83-9e94-4dfd-8a9f-2d9d5ad61a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.800604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba572a83-9e94-4dfd-8a9f-2d9d5ad61a8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.811272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba572a83-9e94-4dfd-8a9f-2d9d5ad61a8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.812795 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba572a83-9e94-4dfd-8a9f-2d9d5ad61a8d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ba572a83-9e94-4dfd-8a9f-2d9d5ad61a8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:40.829998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:40.830808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:40.831644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:40.831644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:40.831644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:40.831644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:40.831644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 251/506] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:40.832216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba572a83-9e94-4dfd-8a9f-2d9d5ad61a8d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.893830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 256/507] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:40.894491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fc23d-b4c0-42cf-bc03-55fc6185ad89 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.932307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba572a83-9e94-4dfd-8a9f-2d9d5ad61a8d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.935101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.964942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef3175-c19d-4e74-ba20-f22c74b17c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.965728 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef3175-c19d-4e74-ba20-f22c74b17c23 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.966009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef3175-c19d-4e74-ba20-f22c74b17c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.966286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef3175-c19d-4e74-ba20-f22c74b17c23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.977145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef3175-c19d-4e74-ba20-f22c74b17c23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.978599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef3175-c19d-4e74-ba20-f22c74b17c23 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.994168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fc23d-b4c0-42cf-bc03-55fc6185ad89 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b2ef3175-c19d-4e74-ba20-f22c74b17c23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:40.995501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:40.996376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:40.997206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:40.997206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:40.997206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:40.997206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:40.997206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 252/508] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:40.997531 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef3175-c19d-4e74-ba20-f22c74b17c23 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.016123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.033311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d7b70-eed0-46d3-8a4d-ec4519be4cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.034100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d7b70-eed0-46d3-8a4d-ec4519be4cbd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.034379 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d7b70-eed0-46d3-8a4d-ec4519be4cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.034653 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d7b70-eed0-46d3-8a4d-ec4519be4cbd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.041532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d7b70-eed0-46d3-8a4d-ec4519be4cbd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.042975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d7b70-eed0-46d3-8a4d-ec4519be4cbd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-771d7b70-eed0-46d3-8a4d-ec4519be4cbd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:41.058156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:41.059045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:41.059970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:41.059970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:41.059970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:41.059970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:41.059970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:41.059970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:41.059970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:41.059970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:41.059970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:49:41.059970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:41.059970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 257/509] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:41.060525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d7b70-eed0-46d3-8a4d-ec4519be4cbd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.102596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef3175-c19d-4e74-ba20-f22c74b17c23 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.136176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab97d2-4d69-466b-9587-cb65c8bbb5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.136858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab97d2-4d69-466b-9587-cb65c8bbb5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.137136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab97d2-4d69-466b-9587-cb65c8bbb5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.137410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab97d2-4d69-466b-9587-cb65c8bbb5c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.138126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab97d2-4d69-466b-9587-cb65c8bbb5c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:41.138431 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab97d2-4d69-466b-9587-cb65c8bbb5c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.140439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab97d2-4d69-466b-9587-cb65c8bbb5c7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.151634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 253/510] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:41.152487 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab97d2-4d69-466b-9587-cb65c8bbb5c7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.157759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d7b70-eed0-46d3-8a4d-ec4519be4cbd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.159322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.191924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64ac83-9fd7-46e1-a42e-dc85a35a4607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.192588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64ac83-9fd7-46e1-a42e-dc85a35a4607 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.192874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64ac83-9fd7-46e1-a42e-dc85a35a4607 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.193154 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64ac83-9fd7-46e1-a42e-dc85a35a4607 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.193882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64ac83-9fd7-46e1-a42e-dc85a35a4607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:41.194193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64ac83-9fd7-46e1-a42e-dc85a35a4607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.195567 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64ac83-9fd7-46e1-a42e-dc85a35a4607 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.207034 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 258/511] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:41.207850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64ac83-9fd7-46e1-a42e-dc85a35a4607 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.228951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.252248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab97d2-4d69-466b-9587-cb65c8bbb5c7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.284997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac203c-3ba5-4a4d-bda7-7f5c84f6314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.285794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac203c-3ba5-4a4d-bda7-7f5c84f6314f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.286073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac203c-3ba5-4a4d-bda7-7f5c84f6314f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.286345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac203c-3ba5-4a4d-bda7-7f5c84f6314f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.296666 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac203c-3ba5-4a4d-bda7-7f5c84f6314f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.298086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac203c-3ba5-4a4d-bda7-7f5c84f6314f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.307968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64ac83-9fd7-46e1-a42e-dc85a35a4607 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1fac203c-3ba5-4a4d-bda7-7f5c84f6314f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:41.315409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:41.316424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:41.317324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:41.317324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:41.317324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:41.317324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:41.317324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 254/512] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:41.317703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac203c-3ba5-4a4d-bda7-7f5c84f6314f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.346405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5857b6-2cde-4c77-bbfe-2042fff91471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.347001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5857b6-2cde-4c77-bbfe-2042fff91471 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.347287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5857b6-2cde-4c77-bbfe-2042fff91471 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.347569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5857b6-2cde-4c77-bbfe-2042fff91471 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.348331 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5857b6-2cde-4c77-bbfe-2042fff91471 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:41.348642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5857b6-2cde-4c77-bbfe-2042fff91471 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.350014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5857b6-2cde-4c77-bbfe-2042fff91471 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.362497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 259/513] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:41.363344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5857b6-2cde-4c77-bbfe-2042fff91471 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.383954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.416115 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac203c-3ba5-4a4d-bda7-7f5c84f6314f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.441918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.450436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fe738-78dc-4e75-acd9-636077ae7fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.451103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fe738-78dc-4e75-acd9-636077ae7fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.451387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fe738-78dc-4e75-acd9-636077ae7fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.451717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fe738-78dc-4e75-acd9-636077ae7fc1 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.452556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fe738-78dc-4e75-acd9-636077ae7fc1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:41.452870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fe738-78dc-4e75-acd9-636077ae7fc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.454291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fe738-78dc-4e75-acd9-636077ae7fc1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.463040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5857b6-2cde-4c77-bbfe-2042fff91471 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.470701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 255/514] 10.222.0.65 () {56 vars in 1137 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:41.471544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fe738-78dc-4e75-acd9-636077ae7fc1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.500333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b548b7d-a816-4ced-844a-997601dc3adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.501127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b548b7d-a816-4ced-844a-997601dc3adb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.501412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b548b7d-a816-4ced-844a-997601dc3adb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.501692 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b548b7d-a816-4ced-844a-997601dc3adb None admin] PATH_INFO: `/v3/users/glance` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.508695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b548b7d-a816-4ced-844a-997601dc3adb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.510155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b548b7d-a816-4ced-844a-997601dc3adb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9b548b7d-a816-4ced-844a-997601dc3adb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:41.524443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:41.525499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:41.526509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:41.526509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:41.526509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:41.526509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:41.526509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:41.526509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:41.526509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:41.526509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:41.526509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 10:49:41.526509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:41.526509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 260/515] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:41.526509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b548b7d-a816-4ced-844a-997601dc3adb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.572802 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fe738-78dc-4e75-acd9-636077ae7fc1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.601767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.606122 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83f325-26f1-4f86-b73b-d831789f0206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.606911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83f325-26f1-4f86-b73b-d831789f0206 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.607191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83f325-26f1-4f86-b73b-d831789f0206 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.607460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83f325-26f1-4f86-b73b-d831789f0206 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.615203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83f325-26f1-4f86-b73b-d831789f0206 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.616660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83f325-26f1-4f86-b73b-d831789f0206 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.624177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b548b7d-a816-4ced-844a-997601dc3adb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4a83f325-26f1-4f86-b73b-d831789f0206 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:41.631830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:41.632893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:41.638612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:41.638612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:41.638612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:41.638612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:41.638612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:41.638612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:41.638612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:41.638612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:41.638612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 10:49:41.638612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:41.638612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 256/516] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:41.638612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83f325-26f1-4f86-b73b-d831789f0206 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.658242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db6052-beac-4efe-9234-3c44387e77f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.658900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db6052-beac-4efe-9234-3c44387e77f9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.659205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db6052-beac-4efe-9234-3c44387e77f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.659502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db6052-beac-4efe-9234-3c44387e77f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.660347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db6052-beac-4efe-9234-3c44387e77f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:41.660648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db6052-beac-4efe-9234-3c44387e77f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.662009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db6052-beac-4efe-9234-3c44387e77f9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.673456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 261/517] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:41.674256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db6052-beac-4efe-9234-3c44387e77f9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.681773 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.735113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83f325-26f1-4f86-b73b-d831789f0206 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.768089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694708c-64c1-448a-b6b7-a98f4591aec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.768754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694708c-64c1-448a-b6b7-a98f4591aec9 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.769037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694708c-64c1-448a-b6b7-a98f4591aec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.769337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694708c-64c1-448a-b6b7-a98f4591aec9 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.770041 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694708c-64c1-448a-b6b7-a98f4591aec9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.770458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db6052-beac-4efe-9234-3c44387e77f9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.771469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694708c-64c1-448a-b6b7-a98f4591aec9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.791181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 257/518] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:41 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:41.792005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694708c-64c1-448a-b6b7-a98f4591aec9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.806864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a707d82-87f2-49ce-adb7-9b060e90a2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.807546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a707d82-87f2-49ce-adb7-9b060e90a2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.807848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a707d82-87f2-49ce-adb7-9b060e90a2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.808128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a707d82-87f2-49ce-adb7-9b060e90a2f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.818458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.885891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a707d82-87f2-49ce-adb7-9b060e90a2f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.887279 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a707d82-87f2-49ce-adb7-9b060e90a2f0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.923239 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694708c-64c1-448a-b6b7-a98f4591aec9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.925133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.964678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f6a47-a91f-4f7a-ad1c-94bcce709224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.965346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f6a47-a91f-4f7a-ad1c-94bcce709224 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.965625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f6a47-a91f-4f7a-ad1c-94bcce709224 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.965897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f6a47-a91f-4f7a-ad1c-94bcce709224 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.966739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f6a47-a91f-4f7a-ad1c-94bcce709224 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:41.967645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f6a47-a91f-4f7a-ad1c-94bcce709224 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.969097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f6a47-a91f-4f7a-ad1c-94bcce709224 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.990244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 258/519] 10.222.0.65 () {56 vars in 1137 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:41.991002 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f6a47-a91f-4f7a-ad1c-94bcce709224 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.005904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 262/520] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:42.006970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a707d82-87f2-49ce-adb7-9b060e90a2f0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.047841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.100123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f6a47-a91f-4f7a-ad1c-94bcce709224 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.117045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a707d82-87f2-49ce-adb7-9b060e90a2f0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.134144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4739ec2-c062-4e47-bf9d-ff9956c6ca72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.134903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4739ec2-c062-4e47-bf9d-ff9956c6ca72 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.135190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4739ec2-c062-4e47-bf9d-ff9956c6ca72 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.135461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4739ec2-c062-4e47-bf9d-ff9956c6ca72 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.136386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4739ec2-c062-4e47-bf9d-ff9956c6ca72 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:42.136640 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4739ec2-c062-4e47-bf9d-ff9956c6ca72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.138086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4739ec2-c062-4e47-bf9d-ff9956c6ca72 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.153192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383de4b-2bcb-4e5a-bc9c-49105c1a753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.154016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383de4b-2bcb-4e5a-bc9c-49105c1a753b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.154283 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383de4b-2bcb-4e5a-bc9c-49105c1a753b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.154542 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383de4b-2bcb-4e5a-bc9c-49105c1a753b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.161220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 259/521] 10.222.0.65 () {56 vars in 1139 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:42.161220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4739ec2-c062-4e47-bf9d-ff9956c6ca72 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.167606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383de4b-2bcb-4e5a-bc9c-49105c1a753b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.169131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383de4b-2bcb-4e5a-bc9c-49105c1a753b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.177895 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f383de4b-2bcb-4e5a-bc9c-49105c1a753b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:42.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:42.185560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:42.186469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:42.186469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:42.186469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:42.186469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:42.186469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:42.186469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:42.186469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:42.186469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:42.186469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 263/522] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:42.186949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383de4b-2bcb-4e5a-bc9c-49105c1a753b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.268593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4739ec2-c062-4e47-bf9d-ff9956c6ca72 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.292210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.296372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383de4b-2bcb-4e5a-bc9c-49105c1a753b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.309786 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96719e6-ba1f-4703-b18a-201fd285213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.310904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96719e6-ba1f-4703-b18a-201fd285213f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.311367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96719e6-ba1f-4703-b18a-201fd285213f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.312200 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96719e6-ba1f-4703-b18a-201fd285213f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.333229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195f3e4-3f7a-40ca-a26e-d93fabb31333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.334029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195f3e4-3f7a-40ca-a26e-d93fabb31333 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.334311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195f3e4-3f7a-40ca-a26e-d93fabb31333 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.334585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195f3e4-3f7a-40ca-a26e-d93fabb31333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.345844 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195f3e4-3f7a-40ca-a26e-d93fabb31333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.347304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195f3e4-3f7a-40ca-a26e-d93fabb31333 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9195f3e4-3f7a-40ca-a26e-d93fabb31333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:42.364678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:42.365722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:42.366891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:42.366891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:42.366891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:42.366891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:42.370162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 264/523] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:42.370993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195f3e4-3f7a-40ca-a26e-d93fabb31333 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.390953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96719e6-ba1f-4703-b18a-201fd285213f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.392280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96719e6-ba1f-4703-b18a-201fd285213f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.410975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.482330 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195f3e4-3f7a-40ca-a26e-d93fabb31333 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.504332 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 260/524] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:42.504332 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96719e6-ba1f-4703-b18a-201fd285213f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.520039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48cec5-39ce-4177-ac62-c1983d0cd142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.521420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48cec5-39ce-4177-ac62-c1983d0cd142 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.522005 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48cec5-39ce-4177-ac62-c1983d0cd142 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.522457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48cec5-39ce-4177-ac62-c1983d0cd142 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.545462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.600040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48cec5-39ce-4177-ac62-c1983d0cd142 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.601515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48cec5-39ce-4177-ac62-c1983d0cd142 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.607469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96719e6-ba1f-4703-b18a-201fd285213f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.626179 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.643541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c91e5-0eac-406b-ac10-8580cd234130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.644152 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c91e5-0eac-406b-ac10-8580cd234130 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.644432 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c91e5-0eac-406b-ac10-8580cd234130 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.644703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c91e5-0eac-406b-ac10-8580cd234130 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.645488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c91e5-0eac-406b-ac10-8580cd234130 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:42.645799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c91e5-0eac-406b-ac10-8580cd234130 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.647216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c91e5-0eac-406b-ac10-8580cd234130 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.669400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 261/525] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:42.669400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c91e5-0eac-406b-ac10-8580cd234130 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.715494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 265/526] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:42.716855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48cec5-39ce-4177-ac62-c1983d0cd142 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.770288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c91e5-0eac-406b-ac10-8580cd234130 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.775036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.806037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d832d64-018e-432d-b5cb-5933719def3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.806701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d832d64-018e-432d-b5cb-5933719def3f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.806982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d832d64-018e-432d-b5cb-5933719def3f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.807255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d832d64-018e-432d-b5cb-5933719def3f None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.808001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d832d64-018e-432d-b5cb-5933719def3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:42.808307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d832d64-018e-432d-b5cb-5933719def3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.809665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d832d64-018e-432d-b5cb-5933719def3f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.821468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 262/527] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:42.825424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48cec5-39ce-4177-ac62-c1983d0cd142 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.827230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d832d64-018e-432d-b5cb-5933719def3f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.863849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15631489-c26d-4a9b-a3d6-fd44343bb8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.865104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15631489-c26d-4a9b-a3d6-fd44343bb8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.865104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15631489-c26d-4a9b-a3d6-fd44343bb8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.865284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15631489-c26d-4a9b-a3d6-fd44343bb8a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.870286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.877828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15631489-c26d-4a9b-a3d6-fd44343bb8a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.879296 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15631489-c26d-4a9b-a3d6-fd44343bb8a9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-15631489-c26d-4a9b-a3d6-fd44343bb8a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:42.897790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:42.898643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:42.899519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:42.899519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:42.899519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:42.899519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:42.899519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 266/528] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:42.899833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15631489-c26d-4a9b-a3d6-fd44343bb8a9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.943253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d832d64-018e-432d-b5cb-5933719def3f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.977790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135e4bf-44a4-4eba-97b0-1fb9ae6e0812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.978446 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135e4bf-44a4-4eba-97b0-1fb9ae6e0812 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.978730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135e4bf-44a4-4eba-97b0-1fb9ae6e0812 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.979345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135e4bf-44a4-4eba-97b0-1fb9ae6e0812 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.979864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135e4bf-44a4-4eba-97b0-1fb9ae6e0812 None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=b01d875111bd43698bcf99d30f7a7f9a) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:42.980185 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135e4bf-44a4-4eba-97b0-1fb9ae6e0812 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.981617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135e4bf-44a4-4eba-97b0-1fb9ae6e0812 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.995696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 263/529] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/role_assignments?user.id=b01d875111bd43698bcf99d30f7a7f9a&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:42.995696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135e4bf-44a4-4eba-97b0-1fb9ae6e0812 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.021952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15631489-c26d-4a9b-a3d6-fd44343bb8a9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.024267 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.071357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc5b79-adec-418d-b511-363075aee824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.072548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc5b79-adec-418d-b511-363075aee824 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.073001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc5b79-adec-418d-b511-363075aee824 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.073382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc5b79-adec-418d-b511-363075aee824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.089211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc5b79-adec-418d-b511-363075aee824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.090753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc5b79-adec-418d-b511-363075aee824 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.106567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135e4bf-44a4-4eba-97b0-1fb9ae6e0812 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.108818 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-65fc5b79-adec-418d-b511-363075aee824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:43.118349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:43.119278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:43.120158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:43.120158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:43.120158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:43.120158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:43.122264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 267/530] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:43.128752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc5b79-adec-418d-b511-363075aee824 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.144979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e185a-9be2-41bb-a962-5392c486263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.145782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e185a-9be2-41bb-a962-5392c486263a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.146062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e185a-9be2-41bb-a962-5392c486263a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.146339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e185a-9be2-41bb-a962-5392c486263a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.154043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e185a-9be2-41bb-a962-5392c486263a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.155481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e185a-9be2-41bb-a962-5392c486263a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-468e185a-9be2-41bb-a962-5392c486263a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:43.172429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:43.173354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:43.174189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:43.174189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:43.174189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:43.174189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:43.174189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:43.174189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:43.174189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:43.174189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:43.175032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 264/531] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:43.175567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e185a-9be2-41bb-a962-5392c486263a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.243688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc5b79-adec-418d-b511-363075aee824 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.280223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e185a-9be2-41bb-a962-5392c486263a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.281923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a1715-3bd1-488f-ba0a-47037f6f6064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.282945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a1715-3bd1-488f-ba0a-47037f6f6064 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.283448 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a1715-3bd1-488f-ba0a-47037f6f6064 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.283971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a1715-3bd1-488f-ba0a-47037f6f6064 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.284902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a1715-3bd1-488f-ba0a-47037f6f6064 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:43.285409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a1715-3bd1-488f-ba0a-47037f6f6064 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.287103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a1715-3bd1-488f-ba0a-47037f6f6064 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.300610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 268/532] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:43.306343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a1715-3bd1-488f-ba0a-47037f6f6064 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.315665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3517a2a-f606-476d-a83a-5052037203bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.316346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3517a2a-f606-476d-a83a-5052037203bf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.316634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3517a2a-f606-476d-a83a-5052037203bf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.316910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3517a2a-f606-476d-a83a-5052037203bf None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.317631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3517a2a-f606-476d-a83a-5052037203bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:43.317938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3517a2a-f606-476d-a83a-5052037203bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.319306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3517a2a-f606-476d-a83a-5052037203bf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.326889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.335685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 265/533] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:43.335685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3517a2a-f606-476d-a83a-5052037203bf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.358918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.429143 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a1715-3bd1-488f-ba0a-47037f6f6064 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.439020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3517a2a-f606-476d-a83a-5052037203bf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.465094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f53576-5f3d-402b-9267-268780c52f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.465881 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f53576-5f3d-402b-9267-268780c52f47 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.466168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f53576-5f3d-402b-9267-268780c52f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.466449 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f53576-5f3d-402b-9267-268780c52f47 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.467799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f53576-5f3d-402b-9267-268780c52f47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:43.468113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f53576-5f3d-402b-9267-268780c52f47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.469806 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f53576-5f3d-402b-9267-268780c52f47 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.477190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0cd0e-2ce2-4adc-a025-68397a9cf436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.477986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0cd0e-2ce2-4adc-a025-68397a9cf436 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.478316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0cd0e-2ce2-4adc-a025-68397a9cf436 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.478595 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0cd0e-2ce2-4adc-a025-68397a9cf436 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.486512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0cd0e-2ce2-4adc-a025-68397a9cf436 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.488092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0cd0e-2ce2-4adc-a025-68397a9cf436 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.493812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 269/534] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:43.494804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f53576-5f3d-402b-9267-268780c52f47 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d8d0cd0e-2ce2-4adc-a025-68397a9cf436 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:43.502849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:43.503720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:43.504587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:43.504587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:43.504587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:43.504587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:43.504587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:43.504587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:43.504587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:43.504587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:43.504587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 266/535] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:43.505497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0cd0e-2ce2-4adc-a025-68397a9cf436 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.588419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.598226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.617444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f53576-5f3d-402b-9267-268780c52f47 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.624070 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0cd0e-2ce2-4adc-a025-68397a9cf436 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.660040 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ee60e-d425-4191-8927-3bb9a8109d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.660786 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ee60e-d425-4191-8927-3bb9a8109d0f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.661071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ee60e-d425-4191-8927-3bb9a8109d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.661347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ee60e-d425-4191-8927-3bb9a8109d0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.662365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ee60e-d425-4191-8927-3bb9a8109d0f None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=3663612647934677b83fb42aaf06d6b8) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:43.662677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ee60e-d425-4191-8927-3bb9a8109d0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.664225 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ee60e-d425-4191-8927-3bb9a8109d0f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.664439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08c32e-8a50-4af1-b9e6-6af137b87fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.665383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08c32e-8a50-4af1-b9e6-6af137b87fcb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.665809 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08c32e-8a50-4af1-b9e6-6af137b87fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.666380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08c32e-8a50-4af1-b9e6-6af137b87fcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.675704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 267/536] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/role_assignments?user.id=3663612647934677b83fb42aaf06d6b8&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:43.675704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ee60e-d425-4191-8927-3bb9a8109d0f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.684638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08c32e-8a50-4af1-b9e6-6af137b87fcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.686092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08c32e-8a50-4af1-b9e6-6af137b87fcb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2f08c32e-8a50-4af1-b9e6-6af137b87fcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:43.711084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:43.712016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:43.712870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:43.712870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:43.712870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:43.712870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:43.712870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:43.712870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:43.712870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:43.712870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:43.713836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 270/537] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:43.720758 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08c32e-8a50-4af1-b9e6-6af137b87fcb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.877646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ee60e-d425-4191-8927-3bb9a8109d0f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.901198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08c32e-8a50-4af1-b9e6-6af137b87fcb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.920623 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270b501-6518-4d9e-86d8-e1eb93997dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.921287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270b501-6518-4d9e-86d8-e1eb93997dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.921574 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270b501-6518-4d9e-86d8-e1eb93997dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.921856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270b501-6518-4d9e-86d8-e1eb93997dd7 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.922653 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270b501-6518-4d9e-86d8-e1eb93997dd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:43.922968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270b501-6518-4d9e-86d8-e1eb93997dd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.924428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270b501-6518-4d9e-86d8-e1eb93997dd7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.940061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 268/538] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:43.941421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868265fb-401c-43e7-a4b5-cd8e53b3edef None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.941634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868265fb-401c-43e7-a4b5-cd8e53b3edef None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.941928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868265fb-401c-43e7-a4b5-cd8e53b3edef None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.947435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 269/539] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:43.947435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868265fb-401c-43e7-a4b5-cd8e53b3edef None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.953737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc4680-dd8c-4904-a18a-4fd580624f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.953737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc4680-dd8c-4904-a18a-4fd580624f5c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.954117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc4680-dd8c-4904-a18a-4fd580624f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.954470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc4680-dd8c-4904-a18a-4fd580624f5c None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.955342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc4680-dd8c-4904-a18a-4fd580624f5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:43.955795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc4680-dd8c-4904-a18a-4fd580624f5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.957318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc4680-dd8c-4904-a18a-4fd580624f5c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.959356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.961224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.977679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 271/540] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:43.978961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f676c-c1bb-4b5b-8089-0be0fe0635c1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.979258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f676c-c1bb-4b5b-8089-0be0fe0635c1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.979540 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f676c-c1bb-4b5b-8089-0be0fe0635c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.996527 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-471f676c-c1bb-4b5b-8089-0be0fe0635c1 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:43.997813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-471f676c-c1bb-4b5b-8089-0be0fe0635c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:44.072811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 272/541] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:44.072811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f676c-c1bb-4b5b-8089-0be0fe0635c1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:44.105260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868265fb-401c-43e7-a4b5-cd8e53b3edef None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:44.150548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c897b-f5af-4e81-a057-0c2c2ed3ceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:44.151918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c897b-f5af-4e81-a057-0c2c2ed3ceb0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.152628 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c897b-f5af-4e81-a057-0c2c2ed3ceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.153271 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c897b-f5af-4e81-a057-0c2c2ed3ceb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.154313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c897b-f5af-4e81-a057-0c2c2ed3ceb0 None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=5af6cc8c56db48beae2550954446c119) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:44.154871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c897b-f5af-4e81-a057-0c2c2ed3ceb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:44.156535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c897b-f5af-4e81-a057-0c2c2ed3ceb0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:44.166249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 270/542] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/role_assignments?user.id=5af6cc8c56db48beae2550954446c119&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:44.167374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c897b-f5af-4e81-a057-0c2c2ed3ceb0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:44.194344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f676c-c1bb-4b5b-8089-0be0fe0635c1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:44.245271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:44.249785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:44.272433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ec854-8f3b-4518-aa84-588e99c9be04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:44.273116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ec854-8f3b-4518-aa84-588e99c9be04 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.273487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ec854-8f3b-4518-aa84-588e99c9be04 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.273679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ec854-8f3b-4518-aa84-588e99c9be04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.274526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ec854-8f3b-4518-aa84-588e99c9be04 None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=e96b136657864104a51900debd1fbe04) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:44.274845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ec854-8f3b-4518-aa84-588e99c9be04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:44.276353 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ec854-8f3b-4518-aa84-588e99c9be04 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:44.285700 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c897b-f5af-4e81-a057-0c2c2ed3ceb0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:44.290134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 273/543] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/role_assignments?user.id=e96b136657864104a51900debd1fbe04&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:44.325147 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9cf81-5c76-4013-85e3-1514b45a300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:44.326335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9cf81-5c76-4013-85e3-1514b45a300a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.327386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9cf81-5c76-4013-85e3-1514b45a300a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.327911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9cf81-5c76-4013-85e3-1514b45a300a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.340517 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9cf81-5c76-4013-85e3-1514b45a300a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-57d9cf81-5c76-4013-85e3-1514b45a300a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:44.348860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:44.349828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:44.350638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:44.350638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:44.350638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:44.350638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:44.350638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:44.350638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:44.350638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:44.350638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:44.350638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:44.350638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:44.350638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:44.350638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:44.359777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 271/544] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:44.361549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ec854-8f3b-4518-aa84-588e99c9be04 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:44.494848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ec854-8f3b-4518-aa84-588e99c9be04 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:44.532975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed446c97-9a31-464f-987a-bd32f147ae04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:44.533664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed446c97-9a31-464f-987a-bd32f147ae04 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.533952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed446c97-9a31-464f-987a-bd32f147ae04 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.534237 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed446c97-9a31-464f-987a-bd32f147ae04 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.534999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed446c97-9a31-464f-987a-bd32f147ae04 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:44.535309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed446c97-9a31-464f-987a-bd32f147ae04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:44.536743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed446c97-9a31-464f-987a-bd32f147ae04 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:44.547044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 274/545] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:44.552920 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9cf81-5c76-4013-85e3-1514b45a300a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:44.697638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9cf81-5c76-4013-85e3-1514b45a300a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:44.732303 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:44.745480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e046c9b-9f6f-417a-8482-42f26c19ef74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:44.746531 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e046c9b-9f6f-417a-8482-42f26c19ef74 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.746995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e046c9b-9f6f-417a-8482-42f26c19ef74 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.747449 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e046c9b-9f6f-417a-8482-42f26c19ef74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.768039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e046c9b-9f6f-417a-8482-42f26c19ef74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:44.769790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e046c9b-9f6f-417a-8482-42f26c19ef74 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0e046c9b-9f6f-417a-8482-42f26c19ef74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:44.792495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:44.793412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:44.794246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:44.794246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:44.794246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:44.794246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:44.796780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 272/546] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:44.802300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed446c97-9a31-464f-987a-bd32f147ae04 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:44.854487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:44.918233 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed446c97-9a31-464f-987a-bd32f147ae04 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:44.957271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5717dd1-5541-4a82-8ff4-a3938c1bf358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:44.958226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5717dd1-5541-4a82-8ff4-a3938c1bf358 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.958669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5717dd1-5541-4a82-8ff4-a3938c1bf358 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.959088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5717dd1-5541-4a82-8ff4-a3938c1bf358 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.959989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5717dd1-5541-4a82-8ff4-a3938c1bf358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:44.960439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5717dd1-5541-4a82-8ff4-a3938c1bf358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:44.961983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5717dd1-5541-4a82-8ff4-a3938c1bf358 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:44.981973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 275/547] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:44.987903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e046c9b-9f6f-417a-8482-42f26c19ef74 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.121976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e046c9b-9f6f-417a-8482-42f26c19ef74 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:45.156652 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbd54d-94b9-47ca-8e8a-e5a55250be84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:45.157576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbd54d-94b9-47ca-8e8a-e5a55250be84 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.157965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbd54d-94b9-47ca-8e8a-e5a55250be84 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.158336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbd54d-94b9-47ca-8e8a-e5a55250be84 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.165923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbd54d-94b9-47ca-8e8a-e5a55250be84 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:45.167446 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbd54d-94b9-47ca-8e8a-e5a55250be84 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-60dbd54d-94b9-47ca-8e8a-e5a55250be84 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:45.181719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:45.182718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:45.184131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:45.184131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:45.184131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:45.184131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:45.184131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:45.184131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:45.184131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:45.184131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:45.184131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:49:45.184131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:45.187699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 273/548] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:45.189075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5717dd1-5541-4a82-8ff4-a3938c1bf358 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.306609 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5717dd1-5541-4a82-8ff4-a3938c1bf358 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:45.321711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:45.343527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3158c-6d57-482b-91ba-d24cc56e2603 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.343875 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3158c-6d57-482b-91ba-d24cc56e2603 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.344136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3158c-6d57-482b-91ba-d24cc56e2603 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.349288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 274/549] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:45.350808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677c17c-9dd1-48f8-acb3-6d7c44c336cd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.351095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677c17c-9dd1-48f8-acb3-6d7c44c336cd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.351436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677c17c-9dd1-48f8-acb3-6d7c44c336cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.355191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e83b2d-6271-4565-a846-7d920fbeda14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:45.356127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e83b2d-6271-4565-a846-7d920fbeda14 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.356611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e83b2d-6271-4565-a846-7d920fbeda14 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.357230 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e83b2d-6271-4565-a846-7d920fbeda14 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.358723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e83b2d-6271-4565-a846-7d920fbeda14 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:45.359236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e83b2d-6271-4565-a846-7d920fbeda14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:45.360938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e83b2d-6271-4565-a846-7d920fbeda14 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:45.369485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9677c17c-9dd1-48f8-acb3-6d7c44c336cd None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:45.370672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9677c17c-9dd1-48f8-acb3-6d7c44c336cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:45.427154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:45.434953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 276/550] 10.222.0.65 () {56 vars in 1135 bytes} [Thu Jun 23 10:49:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:45.440210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e83b2d-6271-4565-a846-7d920fbeda14 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.490057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 275/551] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:45.513768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677c17c-9dd1-48f8-acb3-6d7c44c336cd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.629817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e83b2d-6271-4565-a846-7d920fbeda14 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:45.663894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677c17c-9dd1-48f8-acb3-6d7c44c336cd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:45.707391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab8e30-6669-45ce-8d0b-fc543c2228f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:45.707618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415f1da-5bf7-4e6b-882b-297b5d2a9f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:45.708100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415f1da-5bf7-4e6b-882b-297b5d2a9f83 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.708223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab8e30-6669-45ce-8d0b-fc543c2228f2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.708397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415f1da-5bf7-4e6b-882b-297b5d2a9f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.708490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab8e30-6669-45ce-8d0b-fc543c2228f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.708666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415f1da-5bf7-4e6b-882b-297b5d2a9f83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.708761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab8e30-6669-45ce-8d0b-fc543c2228f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.728718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab8e30-6669-45ce-8d0b-fc543c2228f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0aab8e30-6669-45ce-8d0b-fc543c2228f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:45.741037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:45.742006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:45.742922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:45.742922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:45.742922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:45.742922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:45.742922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:45.742922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:45.742922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:45.742922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:45.742922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:45.742922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:45.742922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:45.742922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:45.750345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 276/552] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:45.754285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab8e30-6669-45ce-8d0b-fc543c2228f2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.780900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:45.823804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:45.838110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415f1da-5bf7-4e6b-882b-297b5d2a9f83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:45.839596 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415f1da-5bf7-4e6b-882b-297b5d2a9f83 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:45.897413 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab8e30-6669-45ce-8d0b-fc543c2228f2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:45.932246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39d95c-6596-49ee-b31f-a1fb0ee2f1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:45.932910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39d95c-6596-49ee-b31f-a1fb0ee2f1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.933190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39d95c-6596-49ee-b31f-a1fb0ee2f1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.933464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39d95c-6596-49ee-b31f-a1fb0ee2f1d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.934209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39d95c-6596-49ee-b31f-a1fb0ee2f1d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:45.934516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39d95c-6596-49ee-b31f-a1fb0ee2f1d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:45.935953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39d95c-6596-49ee-b31f-a1fb0ee2f1d5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:45.944609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 277/553] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:45.952973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39d95c-6596-49ee-b31f-a1fb0ee2f1d5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.004619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 277/554] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:46.006316 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c8d51-c9ce-44c9-810e-8924acfe1783 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.006773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c8d51-c9ce-44c9-810e-8924acfe1783 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.007060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c8d51-c9ce-44c9-810e-8924acfe1783 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.008596 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 278/555] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:46.011724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c8d51-c9ce-44c9-810e-8924acfe1783 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.070376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.079165 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39d95c-6596-49ee-b31f-a1fb0ee2f1d5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.088827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.122515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4ea5c-b31b-4018-ae1a-608cabeead1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.123365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4ea5c-b31b-4018-ae1a-608cabeead1e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.123648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4ea5c-b31b-4018-ae1a-608cabeead1e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.123946 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4ea5c-b31b-4018-ae1a-608cabeead1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.130513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c8d51-c9ce-44c9-810e-8924acfe1783 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.134706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4ea5c-b31b-4018-ae1a-608cabeead1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.136176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4ea5c-b31b-4018-ae1a-608cabeead1e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-80d4ea5c-b31b-4018-ae1a-608cabeead1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:46.153970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:46.155627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:46.161037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:46.161037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:46.161037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:46.161037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:46.161037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 278/556] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:46.161037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ff3c6-b761-445f-ba08-0d1e16636979 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.161037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ff3c6-b761-445f-ba08-0d1e16636979 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.161037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ff3c6-b761-445f-ba08-0d1e16636979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.175571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fa1c5-353c-4296-9ed5-b4368151564f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.176570 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fa1c5-353c-4296-9ed5-b4368151564f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.176982 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fa1c5-353c-4296-9ed5-b4368151564f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.177246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fa1c5-353c-4296-9ed5-b4368151564f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.181765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-886ff3c6-b761-445f-ba08-0d1e16636979 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:46.183304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-886ff3c6-b761-445f-ba08-0d1e16636979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:46.189604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fa1c5-353c-4296-9ed5-b4368151564f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.191075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fa1c5-353c-4296-9ed5-b4368151564f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a24fa1c5-353c-4296-9ed5-b4368151564f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:46.214499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:46.216043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:46.217938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:46.217938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:46.217938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:46.217938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:46.219730 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 279/557] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:46.220913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fa1c5-353c-4296-9ed5-b4368151564f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.271970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 279/558] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:46.272871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ff3c6-b761-445f-ba08-0d1e16636979 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.328452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fa1c5-353c-4296-9ed5-b4368151564f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.349460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.366502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a5cca-7373-4d4d-a1b3-513d43d4340c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.367542 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a5cca-7373-4d4d-a1b3-513d43d4340c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.368111 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a5cca-7373-4d4d-a1b3-513d43d4340c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.368603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a5cca-7373-4d4d-a1b3-513d43d4340c None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.369529 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a5cca-7373-4d4d-a1b3-513d43d4340c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:46.370037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a5cca-7373-4d4d-a1b3-513d43d4340c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.370301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.371843 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a5cca-7373-4d4d-a1b3-513d43d4340c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.390428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 280/559] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:46.391606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a5cca-7373-4d4d-a1b3-513d43d4340c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.395711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ff3c6-b761-445f-ba08-0d1e16636979 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.430004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01dc34-9d9c-48ea-a9c9-c325e8b5dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.430676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01dc34-9d9c-48ea-a9c9-c325e8b5dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.430957 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01dc34-9d9c-48ea-a9c9-c325e8b5dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.431233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01dc34-9d9c-48ea-a9c9-c325e8b5dfc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.432033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01dc34-9d9c-48ea-a9c9-c325e8b5dfc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:46.432390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01dc34-9d9c-48ea-a9c9-c325e8b5dfc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.433755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01dc34-9d9c-48ea-a9c9-c325e8b5dfc6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.451055 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 280/560] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:46.452573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01dc34-9d9c-48ea-a9c9-c325e8b5dfc6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.499792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a5cca-7373-4d4d-a1b3-513d43d4340c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.537182 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace5e0c-1ae7-4098-83b6-3c00fa23c50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.537981 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace5e0c-1ae7-4098-83b6-3c00fa23c50a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.538264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace5e0c-1ae7-4098-83b6-3c00fa23c50a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.538535 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace5e0c-1ae7-4098-83b6-3c00fa23c50a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.554728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace5e0c-1ae7-4098-83b6-3c00fa23c50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:46.558011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01dc34-9d9c-48ea-a9c9-c325e8b5dfc6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-aace5e0c-1ae7-4098-83b6-3c00fa23c50a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:46.565381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:46.566304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:46.567146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:46.567146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:46.567146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:46.567146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:46.567146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:46.567146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:46.567146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:46.567146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:46.567146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:46.567146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:46.567146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:46.567146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:46.572546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 281/561] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:46.572546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace5e0c-1ae7-4098-83b6-3c00fa23c50a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.601376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1660d-3c78-4f58-ba46-1713211cb84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.602451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1660d-3c78-4f58-ba46-1713211cb84b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.603039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1660d-3c78-4f58-ba46-1713211cb84b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.603389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1660d-3c78-4f58-ba46-1713211cb84b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.608581 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.617999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1660d-3c78-4f58-ba46-1713211cb84b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.619683 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1660d-3c78-4f58-ba46-1713211cb84b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.636190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-50b1660d-3c78-4f58-ba46-1713211cb84b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:46.637838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:46.638874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:46.639861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:46.639861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:46.639861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:46.639861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:46.639861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:46.639861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:46.639861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:46.639861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:46.639861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:49:46.639861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:46.643895 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 281/562] 10.222.0.65 () {56 vars in 1093 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:46.644712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1660d-3c78-4f58-ba46-1713211cb84b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.712773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace5e0c-1ae7-4098-83b6-3c00fa23c50a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.749589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1660d-3c78-4f58-ba46-1713211cb84b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.752063 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef970a48-60c4-4c65-87c2-37ea2cbd2ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.753093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef970a48-60c4-4c65-87c2-37ea2cbd2ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.753603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef970a48-60c4-4c65-87c2-37ea2cbd2ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.754079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef970a48-60c4-4c65-87c2-37ea2cbd2ca7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.761919 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef970a48-60c4-4c65-87c2-37ea2cbd2ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.763909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef970a48-60c4-4c65-87c2-37ea2cbd2ca7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ef970a48-60c4-4c65-87c2-37ea2cbd2ca7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:46.779055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:46.780021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:46.780893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:46.780893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:46.780893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:46.780893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:46.780893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:46.780893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:46.780893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:46.780893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:46.780893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 282/563] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:46.781855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef970a48-60c4-4c65-87c2-37ea2cbd2ca7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.785824 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370d57c-a8c1-4964-87e7-b0ec8485d9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.786675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370d57c-a8c1-4964-87e7-b0ec8485d9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.786977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370d57c-a8c1-4964-87e7-b0ec8485d9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.787397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370d57c-a8c1-4964-87e7-b0ec8485d9b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.788047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370d57c-a8c1-4964-87e7-b0ec8485d9b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:46.788359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370d57c-a8c1-4964-87e7-b0ec8485d9b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.789720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370d57c-a8c1-4964-87e7-b0ec8485d9b7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.798489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 282/564] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:46.799750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049a87e-c58e-41cd-a668-d0eb5d8a43fd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.800043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049a87e-c58e-41cd-a668-d0eb5d8a43fd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.800325 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049a87e-c58e-41cd-a668-d0eb5d8a43fd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.801554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 283/565] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:46.802253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049a87e-c58e-41cd-a668-d0eb5d8a43fd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.860749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.883246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.886553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef970a48-60c4-4c65-87c2-37ea2cbd2ca7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.905989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049a87e-c58e-41cd-a668-d0eb5d8a43fd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.920499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877a4c-d3e4-485f-9510-76a0f0563ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.921184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877a4c-d3e4-485f-9510-76a0f0563ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.921468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877a4c-d3e4-485f-9510-76a0f0563ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.921744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877a4c-d3e4-485f-9510-76a0f0563ec6 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.922579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877a4c-d3e4-485f-9510-76a0f0563ec6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:46.922887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877a4c-d3e4-485f-9510-76a0f0563ec6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.924350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877a4c-d3e4-485f-9510-76a0f0563ec6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.939646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d709ff-1a76-420b-97a8-7341be249afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.940331 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d709ff-1a76-420b-97a8-7341be249afc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.940617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d709ff-1a76-420b-97a8-7341be249afc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.940895 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d709ff-1a76-420b-97a8-7341be249afc None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.941635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 283/566] 10.222.0.65 () {56 vars in 1133 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:46.941750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d709ff-1a76-420b-97a8-7341be249afc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:46.942030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d709ff-1a76-420b-97a8-7341be249afc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.942856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e7b20-78a0-449c-8fc1-1da22e22c4cf None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.943146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e7b20-78a0-449c-8fc1-1da22e22c4cf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.943424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e7b20-78a0-449c-8fc1-1da22e22c4cf None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.943512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d709ff-1a76-420b-97a8-7341be249afc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.944853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 284/567] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:46.948519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e7b20-78a0-449c-8fc1-1da22e22c4cf None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.959989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 284/568] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:46.959989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d39ff0-50dd-4873-b9c9-bad8a767be6c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.959989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d39ff0-50dd-4873-b9c9-bad8a767be6c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.959989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d39ff0-50dd-4873-b9c9-bad8a767be6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.972748 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d7d39ff0-50dd-4873-b9c9-bad8a767be6c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:46.973882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d7d39ff0-50dd-4873-b9c9-bad8a767be6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:47.051265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 285/569] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:47.067863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d39ff0-50dd-4873-b9c9-bad8a767be6c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.067959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e7b20-78a0-449c-8fc1-1da22e22c4cf None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.105857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863f11a-07a9-44f7-8fa6-4192ab64192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.106757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863f11a-07a9-44f7-8fa6-4192ab64192e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.108172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863f11a-07a9-44f7-8fa6-4192ab64192e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.108172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863f11a-07a9-44f7-8fa6-4192ab64192e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.118392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863f11a-07a9-44f7-8fa6-4192ab64192e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:47.119878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863f11a-07a9-44f7-8fa6-4192ab64192e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:47.123990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e863f11a-07a9-44f7-8fa6-4192ab64192e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:47.139706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:47.140619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:47.141496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:47.141496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:47.141496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:47.141496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:47.143793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 285/570] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:47.145172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b2aca-342d-4749-a9e7-bf2b85c7eb75 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.145461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b2aca-342d-4749-a9e7-bf2b85c7eb75 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.145745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b2aca-342d-4749-a9e7-bf2b85c7eb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.163604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-536b2aca-342d-4749-a9e7-bf2b85c7eb75 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:47.164860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-536b2aca-342d-4749-a9e7-bf2b85c7eb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:47.167659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:47.181227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d39ff0-50dd-4873-b9c9-bad8a767be6c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.219048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e71bd7-e2d2-4b25-b8ae-248bcf4e6c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.219858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e71bd7-e2d2-4b25-b8ae-248bcf4e6c6e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.220140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e71bd7-e2d2-4b25-b8ae-248bcf4e6c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.220410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e71bd7-e2d2-4b25-b8ae-248bcf4e6c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.231052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e71bd7-e2d2-4b25-b8ae-248bcf4e6c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:47.232467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e71bd7-e2d2-4b25-b8ae-248bcf4e6c6e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:47.238500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 286/571] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:47.239478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b2aca-342d-4749-a9e7-bf2b85c7eb75 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-07e71bd7-e2d2-4b25-b8ae-248bcf4e6c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:47.263602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:47.264589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:47.265477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:47.265477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:47.265477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:47.265477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:47.265477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 286/572] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:47.265966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e71bd7-e2d2-4b25-b8ae-248bcf4e6c6e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.355516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b2aca-342d-4749-a9e7-bf2b85c7eb75 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.373426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:47.376644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e71bd7-e2d2-4b25-b8ae-248bcf4e6c6e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.390946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff24c0-b153-4c8d-820f-4ac92e7a1c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.391605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff24c0-b153-4c8d-820f-4ac92e7a1c74 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.391934 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff24c0-b153-4c8d-820f-4ac92e7a1c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.392214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff24c0-b153-4c8d-820f-4ac92e7a1c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.393046 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff24c0-b153-4c8d-820f-4ac92e7a1c74 None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=d3d8f8acd0484857b451877baf03e895) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:47.393359 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff24c0-b153-4c8d-820f-4ac92e7a1c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:47.394811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff24c0-b153-4c8d-820f-4ac92e7a1c74 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:47.400183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 287/573] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/role_assignments?user.id=d3d8f8acd0484857b451877baf03e895&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:47.400960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff24c0-b153-4c8d-820f-4ac92e7a1c74 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.404261 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:47.426337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d36c09-aa0d-489b-817d-5448ce90ea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.427300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d36c09-aa0d-489b-817d-5448ce90ea47 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.427483 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d36c09-aa0d-489b-817d-5448ce90ea47 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.427747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d36c09-aa0d-489b-817d-5448ce90ea47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.440690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d36c09-aa0d-489b-817d-5448ce90ea47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a6d36c09-aa0d-489b-817d-5448ce90ea47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:47.448656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:47.449909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 287/574] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:47.450755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d36c09-aa0d-489b-817d-5448ce90ea47 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.524141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff24c0-b153-4c8d-820f-4ac92e7a1c74 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.568431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0626f-9617-4022-bcfd-ea1dc807dac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.569106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0626f-9617-4022-bcfd-ea1dc807dac3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.569388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0626f-9617-4022-bcfd-ea1dc807dac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.569663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0626f-9617-4022-bcfd-ea1dc807dac3 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.570382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0626f-9617-4022-bcfd-ea1dc807dac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:47.570687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0626f-9617-4022-bcfd-ea1dc807dac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:47.572096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0626f-9617-4022-bcfd-ea1dc807dac3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:47.572988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d36c09-aa0d-489b-817d-5448ce90ea47 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.585721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 288/575] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:47.586553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0626f-9617-4022-bcfd-ea1dc807dac3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.613458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa954a-3e60-4808-817d-ae285aaf021b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.614258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa954a-3e60-4808-817d-ae285aaf021b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.614548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa954a-3e60-4808-817d-ae285aaf021b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.614832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa954a-3e60-4808-817d-ae285aaf021b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.624991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa954a-3e60-4808-817d-ae285aaf021b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-22fa954a-3e60-4808-817d-ae285aaf021b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:47.632719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:47.633645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:47.634495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:47.634495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:47.634495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:47.634495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:47.634495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:47.634495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:47.634495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:47.634495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:47.634495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:47.634495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:47.634495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:47.634495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:47.639428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 288/576] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:47.640151 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa954a-3e60-4808-817d-ae285aaf021b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.658871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:47.662631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:47.700265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0626f-9617-4022-bcfd-ea1dc807dac3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.736304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88050c6f-b985-406d-96d5-31b814050290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.736976 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88050c6f-b985-406d-96d5-31b814050290 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.737269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88050c6f-b985-406d-96d5-31b814050290 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.737547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88050c6f-b985-406d-96d5-31b814050290 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.738334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88050c6f-b985-406d-96d5-31b814050290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:47.738649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88050c6f-b985-406d-96d5-31b814050290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:47.740050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88050c6f-b985-406d-96d5-31b814050290 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:47.744502 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa954a-3e60-4808-817d-ae285aaf021b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.751765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 289/577] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:47.752574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88050c6f-b985-406d-96d5-31b814050290 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.784360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9082a-290b-4a8a-96f9-2376c7cfecd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.785368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9082a-290b-4a8a-96f9-2376c7cfecd2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.785809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9082a-290b-4a8a-96f9-2376c7cfecd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.786228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9082a-290b-4a8a-96f9-2376c7cfecd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.787112 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9082a-290b-4a8a-96f9-2376c7cfecd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:47.787703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9082a-290b-4a8a-96f9-2376c7cfecd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:47.789350 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9082a-290b-4a8a-96f9-2376c7cfecd2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:47.797799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 289/578] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:47.798888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9082a-290b-4a8a-96f9-2376c7cfecd2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.862349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88050c6f-b985-406d-96d5-31b814050290 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.886594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:47.890173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:47.899643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08805594-696e-4f54-b482-f3687b7e6798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.900658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08805594-696e-4f54-b482-f3687b7e6798 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.900955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08805594-696e-4f54-b482-f3687b7e6798 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.901229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08805594-696e-4f54-b482-f3687b7e6798 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.908964 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08805594-696e-4f54-b482-f3687b7e6798 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:47.910163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9082a-290b-4a8a-96f9-2376c7cfecd2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.910398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08805594-696e-4f54-b482-f3687b7e6798 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-08805594-696e-4f54-b482-f3687b7e6798 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:47.924515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:47.925484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:47.926424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:47.926424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:47.926424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:47.926424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:47.926424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:47.926424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:47.926424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:47.926424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:47.926424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:49:47.926424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:47.926424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 290/579] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:47.927003 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08805594-696e-4f54-b482-f3687b7e6798 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.950932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1708c8-0c74-433c-8e08-61f749bd0b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.952181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1708c8-0c74-433c-8e08-61f749bd0b4f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.952465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1708c8-0c74-433c-8e08-61f749bd0b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.952741 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1708c8-0c74-433c-8e08-61f749bd0b4f None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.953486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1708c8-0c74-433c-8e08-61f749bd0b4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:47.953797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1708c8-0c74-433c-8e08-61f749bd0b4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:47.955149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1708c8-0c74-433c-8e08-61f749bd0b4f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:47.962945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 290/580] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:47.963738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1708c8-0c74-433c-8e08-61f749bd0b4f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:48.033937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08805594-696e-4f54-b482-f3687b7e6798 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:48.067634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551fdba-551a-4a2a-a41b-1d874386c633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:48.068393 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551fdba-551a-4a2a-a41b-1d874386c633 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:48.068952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551fdba-551a-4a2a-a41b-1d874386c633 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:48.068952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551fdba-551a-4a2a-a41b-1d874386c633 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:48.071571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1708c8-0c74-433c-8e08-61f749bd0b4f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:48.075841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551fdba-551a-4a2a-a41b-1d874386c633 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:48.077882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551fdba-551a-4a2a-a41b-1d874386c633 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1551fdba-551a-4a2a-a41b-1d874386c633 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:48.093452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:48.094360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:48.099882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:48.099882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:48.099882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:48.099882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:48.099882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:48.099882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:48.099882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:48.099882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:48.099882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 291/581] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:48.099882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551fdba-551a-4a2a-a41b-1d874386c633 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:48.127463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a812a98-2bf5-4ddc-93a5-0cd7fd05efeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:48.128390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a812a98-2bf5-4ddc-93a5-0cd7fd05efeb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:48.128737 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a812a98-2bf5-4ddc-93a5-0cd7fd05efeb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:48.129075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a812a98-2bf5-4ddc-93a5-0cd7fd05efeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:48.140561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a812a98-2bf5-4ddc-93a5-0cd7fd05efeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:48.141995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a812a98-2bf5-4ddc-93a5-0cd7fd05efeb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:48.144678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:48.148422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4a812a98-2bf5-4ddc-93a5-0cd7fd05efeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:48.160128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:48.161024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:48.161906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:48.161906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:48.161906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:48.161906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:48.161906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 291/582] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:48.162175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a812a98-2bf5-4ddc-93a5-0cd7fd05efeb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:48.211582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551fdba-551a-4a2a-a41b-1d874386c633 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:48.245639 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14e4e5-9eb3-452e-b4ab-0096b8cdd31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:48.246302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14e4e5-9eb3-452e-b4ab-0096b8cdd31c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:48.246585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14e4e5-9eb3-452e-b4ab-0096b8cdd31c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:48.246857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14e4e5-9eb3-452e-b4ab-0096b8cdd31c None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:48.247706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14e4e5-9eb3-452e-b4ab-0096b8cdd31c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:48.248020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14e4e5-9eb3-452e-b4ab-0096b8cdd31c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:48.249986 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14e4e5-9eb3-452e-b4ab-0096b8cdd31c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:48.264859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a812a98-2bf5-4ddc-93a5-0cd7fd05efeb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:48.267808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 292/583] 10.222.0.65 () {56 vars in 1137 bytes} [Thu Jun 23 10:49:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:48.272666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14e4e5-9eb3-452e-b4ab-0096b8cdd31c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:48.299857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f44ee-ff8e-401a-b896-e5eb88bf1a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:48.300833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f44ee-ff8e-401a-b896-e5eb88bf1a81 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:48.301257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f44ee-ff8e-401a-b896-e5eb88bf1a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:48.301672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f44ee-ff8e-401a-b896-e5eb88bf1a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:48.313736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f44ee-ff8e-401a-b896-e5eb88bf1a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:48.315318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f44ee-ff8e-401a-b896-e5eb88bf1a81 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-111f44ee-ff8e-401a-b896-e5eb88bf1a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:48.332323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:48.333356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:48.334419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:48.334419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:48.334419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:48.334419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:48.336244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 292/584] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:48.337147 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f44ee-ff8e-401a-b896-e5eb88bf1a81 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:48.376795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14e4e5-9eb3-452e-b4ab-0096b8cdd31c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:48.378302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:48.390625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:48.412954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf79cde-9734-4c0f-9f25-b38569ba4c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:48.413695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf79cde-9734-4c0f-9f25-b38569ba4c8f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:48.414044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf79cde-9734-4c0f-9f25-b38569ba4c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:48.414325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf79cde-9734-4c0f-9f25-b38569ba4c8f None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:48.415104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf79cde-9734-4c0f-9f25-b38569ba4c8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:48.415411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf79cde-9734-4c0f-9f25-b38569ba4c8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:48.416892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf79cde-9734-4c0f-9f25-b38569ba4c8f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:48.429147 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 293/585] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:48.430025 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf79cde-9734-4c0f-9f25-b38569ba4c8f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:48.457679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f44ee-ff8e-401a-b896-e5eb88bf1a81 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:48.495515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49dee6-7b72-45de-83f4-a8e13b40fade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:48.496349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49dee6-7b72-45de-83f4-a8e13b40fade None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:48.496732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49dee6-7b72-45de-83f4-a8e13b40fade None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:48.497105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49dee6-7b72-45de-83f4-a8e13b40fade None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:48.497935 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49dee6-7b72-45de-83f4-a8e13b40fade None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:48.498914 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49dee6-7b72-45de-83f4-a8e13b40fade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:48.500398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49dee6-7b72-45de-83f4-a8e13b40fade None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:48.513088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 293/586] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:48.514123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49dee6-7b72-45de-83f4-a8e13b40fade None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:48.540600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf79cde-9734-4c0f-9f25-b38569ba4c8f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:48.576938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a880fb-fa4f-4a69-9e7c-e204374eff04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:48.577868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a880fb-fa4f-4a69-9e7c-e204374eff04 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:48.578242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a880fb-fa4f-4a69-9e7c-e204374eff04 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:48.578615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a880fb-fa4f-4a69-9e7c-e204374eff04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:48.589199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a880fb-fa4f-4a69-9e7c-e204374eff04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:48.590640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a880fb-fa4f-4a69-9e7c-e204374eff04 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-11a880fb-fa4f-4a69-9e7c-e204374eff04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:48.609177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:48.610255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:48.615702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:48.615702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:48.615702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:48.615702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:48.615702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 294/587] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:48.615702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a880fb-fa4f-4a69-9e7c-e204374eff04 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:48.627988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49dee6-7b72-45de-83f4-a8e13b40fade None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:48.628942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:48.639481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:48.667706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e3b8e-c4cd-448b-b8dd-438079f68cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:48.668559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e3b8e-c4cd-448b-b8dd-438079f68cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:48.668901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e3b8e-c4cd-448b-b8dd-438079f68cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:48.669248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e3b8e-c4cd-448b-b8dd-438079f68cc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:48.670578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e3b8e-c4cd-448b-b8dd-438079f68cc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:48.670954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e3b8e-c4cd-448b-b8dd-438079f68cc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:48.672686 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e3b8e-c4cd-448b-b8dd-438079f68cc3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:48.689158 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 294/588] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:48.689158 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e3b8e-c4cd-448b-b8dd-438079f68cc3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:48.722769 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a880fb-fa4f-4a69-9e7c-e204374eff04 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:48.758710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bab053-c10a-42a7-a305-baf12b0a360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:48.759606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bab053-c10a-42a7-a305-baf12b0a360b None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:48.759911 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bab053-c10a-42a7-a305-baf12b0a360b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:48.760191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bab053-c10a-42a7-a305-baf12b0a360b None admin] PATH_INFO: `/v3/projects/8458544c021049c587856343e8001429/users/b01d875111bd43698bcf99d30f7a7f9a/roles/b29c780ef8924837b89e1f06648a9563` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:48.774414 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bab053-c10a-42a7-a305-baf12b0a360b None admin] RBAC: Authorizing `identity:create_grant(project_id=8458544c021049c587856343e8001429, user_id=b01d875111bd43698bcf99d30f7a7f9a, role_id=b29c780ef8924837b89e1f06648a9563)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:48.775875 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bab053-c10a-42a7-a305-baf12b0a360b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:48.788841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 295/589] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:48 2022] PUT /identity/v3/projects/8458544c021049c587856343e8001429/users/b01d875111bd43698bcf99d30f7a7f9a/roles/b29c780ef8924837b89e1f06648a9563 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:48.789626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bab053-c10a-42a7-a305-baf12b0a360b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:48.810775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e3b8e-c4cd-448b-b8dd-438079f68cc3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:48.858296 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08257f3e-9356-4a0e-bd63-6dba1043debe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:48.859120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08257f3e-9356-4a0e-bd63-6dba1043debe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:48.859407 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08257f3e-9356-4a0e-bd63-6dba1043debe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:48.859752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08257f3e-9356-4a0e-bd63-6dba1043debe None admin] PATH_INFO: `/v3/users/glance` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:48.864444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:48.873065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08257f3e-9356-4a0e-bd63-6dba1043debe None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:48.875615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08257f3e-9356-4a0e-bd63-6dba1043debe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-08257f3e-9356-4a0e-bd63-6dba1043debe None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:48.896583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:48.897705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:48.899077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:48.899077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:48.899077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:48.899077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:48.899077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:48.899077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:48.899077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:48.899077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:48.899077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 10:49:48.899077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:48.899077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 295/590] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:48.899689 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08257f3e-9356-4a0e-bd63-6dba1043debe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:48.914603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bab053-c10a-42a7-a305-baf12b0a360b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:48.932877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:48.953057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aee2ff-a118-4826-a6ab-217e9113cff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:48.953743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aee2ff-a118-4826-a6ab-217e9113cff3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:48.954023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aee2ff-a118-4826-a6ab-217e9113cff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:48.954299 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aee2ff-a118-4826-a6ab-217e9113cff3 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:48.955085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aee2ff-a118-4826-a6ab-217e9113cff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:48.955398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aee2ff-a118-4826-a6ab-217e9113cff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:48.956852 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aee2ff-a118-4826-a6ab-217e9113cff3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:48.970341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 296/591] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:48.971788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aee2ff-a118-4826-a6ab-217e9113cff3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:49.044055 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08257f3e-9356-4a0e-bd63-6dba1043debe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:49.082173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7867a6-3ed3-4ad8-9b4b-f2cc692a22d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:49.082971 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7867a6-3ed3-4ad8-9b4b-f2cc692a22d2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:49.083375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7867a6-3ed3-4ad8-9b4b-f2cc692a22d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:49.083668 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7867a6-3ed3-4ad8-9b4b-f2cc692a22d2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:49.087904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aee2ff-a118-4826-a6ab-217e9113cff3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:49.092646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7867a6-3ed3-4ad8-9b4b-f2cc692a22d2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:49.094120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7867a6-3ed3-4ad8-9b4b-f2cc692a22d2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3b7867a6-3ed3-4ad8-9b4b-f2cc692a22d2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:49.110218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:49.111369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:49.112396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:49.112396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:49.112396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:49.112396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:49.112396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:49.112396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:49.112396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:49.112396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:49.112396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 10:49:49.112396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:49.116244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 296/592] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:49.117196 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7867a6-3ed3-4ad8-9b4b-f2cc692a22d2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:49.120595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:49.131853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83159ca-b91d-401c-909d-9b8c0e7e1203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:49.132744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83159ca-b91d-401c-909d-9b8c0e7e1203 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:49.133170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83159ca-b91d-401c-909d-9b8c0e7e1203 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:49.133610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83159ca-b91d-401c-909d-9b8c0e7e1203 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:49.134592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83159ca-b91d-401c-909d-9b8c0e7e1203 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:49.135043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83159ca-b91d-401c-909d-9b8c0e7e1203 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:49.136634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83159ca-b91d-401c-909d-9b8c0e7e1203 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:49.157585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 297/593] 10.222.0.65 () {56 vars in 1137 bytes} [Thu Jun 23 10:49:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:49.158642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83159ca-b91d-401c-909d-9b8c0e7e1203 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:49.222451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:49.237678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7867a6-3ed3-4ad8-9b4b-f2cc692a22d2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:49.260739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83159ca-b91d-401c-909d-9b8c0e7e1203 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:49.275489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36639aa5-d894-4071-8fa2-7115c9fe9ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:49.276984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36639aa5-d894-4071-8fa2-7115c9fe9ddd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:49.277277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36639aa5-d894-4071-8fa2-7115c9fe9ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:49.277556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36639aa5-d894-4071-8fa2-7115c9fe9ddd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:49.287230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36639aa5-d894-4071-8fa2-7115c9fe9ddd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:49.289001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36639aa5-d894-4071-8fa2-7115c9fe9ddd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:49.300909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06243090-2a85-4462-bda6-15aab410cf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:49.301587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06243090-2a85-4462-bda6-15aab410cf1a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:49.301945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06243090-2a85-4462-bda6-15aab410cf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:49.302272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06243090-2a85-4462-bda6-15aab410cf1a None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:49.303120 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06243090-2a85-4462-bda6-15aab410cf1a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:49.303477 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06243090-2a85-4462-bda6-15aab410cf1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:49.304977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06243090-2a85-4462-bda6-15aab410cf1a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-36639aa5-d894-4071-8fa2-7115c9fe9ddd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:49.307916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:49.308775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:49.309655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:49.309655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:49.309655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:49.309655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:49.309655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:49.309655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:49.309655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:49.309655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:49.312715 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 297/594] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:49.313692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36639aa5-d894-4071-8fa2-7115c9fe9ddd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:49.335836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 298/595] 10.222.0.65 () {56 vars in 1139 bytes} [Thu Jun 23 10:49:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:49.336982 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06243090-2a85-4462-bda6-15aab410cf1a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:49.384548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:49.420448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36639aa5-d894-4071-8fa2-7115c9fe9ddd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:49.446895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06243090-2a85-4462-bda6-15aab410cf1a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:49.456098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f9430-17c1-49db-877d-ce69afdd1023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:49.457123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f9430-17c1-49db-877d-ce69afdd1023 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:49.457627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f9430-17c1-49db-877d-ce69afdd1023 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:49.458107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f9430-17c1-49db-877d-ce69afdd1023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:49.468231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f9430-17c1-49db-877d-ce69afdd1023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:49.469656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f9430-17c1-49db-877d-ce69afdd1023 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:49.477953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:49.482720 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951170f4-1312-493a-941a-c0ee27e91b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:49.483647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951170f4-1312-493a-941a-c0ee27e91b16 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:49.483951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951170f4-1312-493a-941a-c0ee27e91b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:49.484226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951170f4-1312-493a-941a-c0ee27e91b16 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:49.485010 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951170f4-1312-493a-941a-c0ee27e91b16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:49.485317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951170f4-1312-493a-941a-c0ee27e91b16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:49.486736 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951170f4-1312-493a-941a-c0ee27e91b16 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-424f9430-17c1-49db-877d-ce69afdd1023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:49.488411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:49.489436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:49.490538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:49.490538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:49.490538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:49.490538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:49.492455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 298/596] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:49.493375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f9430-17c1-49db-877d-ce69afdd1023 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:49.501282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 299/597] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:49.507023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a8731-d0eb-470c-8acc-5bb065565394 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:49.507312 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a8731-d0eb-470c-8acc-5bb065565394 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:49.507592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a8731-d0eb-470c-8acc-5bb065565394 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:49.512912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 300/598] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:49.512912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a8731-d0eb-470c-8acc-5bb065565394 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:49.602974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f9430-17c1-49db-877d-ce69afdd1023 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:49.628162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:49.632124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a8731-d0eb-470c-8acc-5bb065565394 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:49.640071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04093df3-7d0d-4336-ae8c-171e6232a714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:49.640810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04093df3-7d0d-4336-ae8c-171e6232a714 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:49.641092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04093df3-7d0d-4336-ae8c-171e6232a714 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:49.641356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04093df3-7d0d-4336-ae8c-171e6232a714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:49.652982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04093df3-7d0d-4336-ae8c-171e6232a714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:49.654750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04093df3-7d0d-4336-ae8c-171e6232a714 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:49.672244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0df274-7e1c-46ef-94ed-04dba67fc3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:49.672907 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0df274-7e1c-46ef-94ed-04dba67fc3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-04093df3-7d0d-4336-ae8c-171e6232a714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:49.673113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:49.674143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:49.675237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:49.675237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:49.675237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:49.675237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:49.675521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0df274-7e1c-46ef-94ed-04dba67fc3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:49.675521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0df274-7e1c-46ef-94ed-04dba67fc3c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:49.675521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0df274-7e1c-46ef-94ed-04dba67fc3c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:49.675521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0df274-7e1c-46ef-94ed-04dba67fc3c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:49.675846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0df274-7e1c-46ef-94ed-04dba67fc3c7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:49.681498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 299/599] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:49.681498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04093df3-7d0d-4336-ae8c-171e6232a714 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:49.695860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 301/600] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:49.697734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b7af1-a850-4eac-82d0-72557aa5394d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:49.698187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b7af1-a850-4eac-82d0-72557aa5394d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:49.698600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b7af1-a850-4eac-82d0-72557aa5394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:49.716080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-556b7af1-a850-4eac-82d0-72557aa5394d None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:49.717311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-556b7af1-a850-4eac-82d0-72557aa5394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:49.732354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:49.785794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04093df3-7d0d-4336-ae8c-171e6232a714 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:49.788787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 302/601] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:49.789185 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b7af1-a850-4eac-82d0-72557aa5394d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:49.840328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919e36b-3807-4250-8d7e-5c7af6f43324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:49.841214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919e36b-3807-4250-8d7e-5c7af6f43324 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:49.841504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919e36b-3807-4250-8d7e-5c7af6f43324 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:49.841804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919e36b-3807-4250-8d7e-5c7af6f43324 None admin] PATH_INFO: `/v3/projects/8458544c021049c587856343e8001429/users/3663612647934677b83fb42aaf06d6b8/roles/b29c780ef8924837b89e1f06648a9563` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:49.856632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919e36b-3807-4250-8d7e-5c7af6f43324 None admin] RBAC: Authorizing `identity:create_grant(project_id=8458544c021049c587856343e8001429, user_id=3663612647934677b83fb42aaf06d6b8, role_id=b29c780ef8924837b89e1f06648a9563)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:49.858865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919e36b-3807-4250-8d7e-5c7af6f43324 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:49.872732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 300/602] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:49 2022] PUT /identity/v3/projects/8458544c021049c587856343e8001429/users/3663612647934677b83fb42aaf06d6b8/roles/b29c780ef8924837b89e1f06648a9563 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:49.874113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919e36b-3807-4250-8d7e-5c7af6f43324 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:49.891605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:49.902313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b7af1-a850-4eac-82d0-72557aa5394d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:49.941842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bd447-bd13-43c3-90fe-231a394820be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:49.942531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bd447-bd13-43c3-90fe-231a394820be None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:49.942869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bd447-bd13-43c3-90fe-231a394820be None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:49.943155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bd447-bd13-43c3-90fe-231a394820be None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:49.943923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bd447-bd13-43c3-90fe-231a394820be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:49.944236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bd447-bd13-43c3-90fe-231a394820be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:49.945706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bd447-bd13-43c3-90fe-231a394820be None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:49.964132 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 303/603] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:49.965183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bd447-bd13-43c3-90fe-231a394820be None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.005548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919e36b-3807-4250-8d7e-5c7af6f43324 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.022885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:50.048115 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b5209-8886-4ec4-a709-c64b6880146c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.048924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b5209-8886-4ec4-a709-c64b6880146c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.049207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b5209-8886-4ec4-a709-c64b6880146c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.049481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b5209-8886-4ec4-a709-c64b6880146c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.057981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b5209-8886-4ec4-a709-c64b6880146c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:50.059436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b5209-8886-4ec4-a709-c64b6880146c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-626b5209-8886-4ec4-a709-c64b6880146c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:50.074119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:50.075082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:50.076016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:50.076016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:50.076016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:50.076016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:50.076016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:50.076016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:50.076016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:50.076016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:50.076016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 301/604] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:50.077081 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b5209-8886-4ec4-a709-c64b6880146c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.088059 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bd447-bd13-43c3-90fe-231a394820be None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.141572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef9f61-dba5-40c7-afbb-9a3140bfb675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.143307 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef9f61-dba5-40c7-afbb-9a3140bfb675 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.144097 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef9f61-dba5-40c7-afbb-9a3140bfb675 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.144579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef9f61-dba5-40c7-afbb-9a3140bfb675 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.159919 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef9f61-dba5-40c7-afbb-9a3140bfb675 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-65ef9f61-dba5-40c7-afbb-9a3140bfb675 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:50.170705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:50.172079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:50.172951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 304/605] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:50.175696 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef9f61-dba5-40c7-afbb-9a3140bfb675 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.196307 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:50.198214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b5209-8886-4ec4-a709-c64b6880146c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.236518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77ac7a-1d49-43ff-8957-e6a2914474e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.238074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77ac7a-1d49-43ff-8957-e6a2914474e8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.238631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77ac7a-1d49-43ff-8957-e6a2914474e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.239214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77ac7a-1d49-43ff-8957-e6a2914474e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.246709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77ac7a-1d49-43ff-8957-e6a2914474e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:50.248945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77ac7a-1d49-43ff-8957-e6a2914474e8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5a77ac7a-1d49-43ff-8957-e6a2914474e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:50.265007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:50.265918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:50.266820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:50.266820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:50.266820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:50.266820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:50.266820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:50.266820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:50.266820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:50.266820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:50.269275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 302/606] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:50.274650 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77ac7a-1d49-43ff-8957-e6a2914474e8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.299953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:50.300580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef9f61-dba5-40c7-afbb-9a3140bfb675 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.343958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b2951-9e4f-449a-8967-b6839c57b7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.344663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b2951-9e4f-449a-8967-b6839c57b7cb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.344949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b2951-9e4f-449a-8967-b6839c57b7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.345229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b2951-9e4f-449a-8967-b6839c57b7cb None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.346026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b2951-9e4f-449a-8967-b6839c57b7cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:50.346396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b2951-9e4f-449a-8967-b6839c57b7cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:50.348074 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b2951-9e4f-449a-8967-b6839c57b7cb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:50.368874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 305/607] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:50.368874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b2951-9e4f-449a-8967-b6839c57b7cb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.394763 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77ac7a-1d49-43ff-8957-e6a2914474e8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.440025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e741d2f-30dd-4e52-8fbd-33ca2ff813a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.441167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e741d2f-30dd-4e52-8fbd-33ca2ff813a3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.441552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e741d2f-30dd-4e52-8fbd-33ca2ff813a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.441930 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e741d2f-30dd-4e52-8fbd-33ca2ff813a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.442958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:50.443789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e741d2f-30dd-4e52-8fbd-33ca2ff813a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:50.444295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e741d2f-30dd-4e52-8fbd-33ca2ff813a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:50.446674 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e741d2f-30dd-4e52-8fbd-33ca2ff813a3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:50.459740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 303/608] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:50.461099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e741d2f-30dd-4e52-8fbd-33ca2ff813a3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.490726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b2951-9e4f-449a-8967-b6839c57b7cb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.537212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa741b02-16f6-4beb-b141-17b8972c7945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.538031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa741b02-16f6-4beb-b141-17b8972c7945 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.538667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa741b02-16f6-4beb-b141-17b8972c7945 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.539063 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa741b02-16f6-4beb-b141-17b8972c7945 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.540573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa741b02-16f6-4beb-b141-17b8972c7945 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:50.540990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa741b02-16f6-4beb-b141-17b8972c7945 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:50.542787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa741b02-16f6-4beb-b141-17b8972c7945 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:50.559879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:50.565157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 306/609] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:50.565157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa741b02-16f6-4beb-b141-17b8972c7945 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.586066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e741d2f-30dd-4e52-8fbd-33ca2ff813a3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.629222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6212768-fd04-4489-b99e-eef5ef9c56a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.630126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6212768-fd04-4489-b99e-eef5ef9c56a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.630412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6212768-fd04-4489-b99e-eef5ef9c56a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.630693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6212768-fd04-4489-b99e-eef5ef9c56a5 None admin] PATH_INFO: `/v3/projects/8458544c021049c587856343e8001429/users/5af6cc8c56db48beae2550954446c119/roles/b29c780ef8924837b89e1f06648a9563` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.651860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6212768-fd04-4489-b99e-eef5ef9c56a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8458544c021049c587856343e8001429, user_id=5af6cc8c56db48beae2550954446c119, role_id=b29c780ef8924837b89e1f06648a9563)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:50.653232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6212768-fd04-4489-b99e-eef5ef9c56a5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:50.678071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 304/610] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:50 2022] PUT /identity/v3/projects/8458544c021049c587856343e8001429/users/5af6cc8c56db48beae2550954446c119/roles/b29c780ef8924837b89e1f06648a9563 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:50.678071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6212768-fd04-4489-b99e-eef5ef9c56a5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.709637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa741b02-16f6-4beb-b141-17b8972c7945 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.809756 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:50.826337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6212768-fd04-4489-b99e-eef5ef9c56a5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.832365 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda5ffc-eacb-408e-8ce0-9322551c048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.833197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda5ffc-eacb-408e-8ce0-9322551c048a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.833482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda5ffc-eacb-408e-8ce0-9322551c048a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.833764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda5ffc-eacb-408e-8ce0-9322551c048a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.852663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda5ffc-eacb-408e-8ce0-9322551c048a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:50.854464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda5ffc-eacb-408e-8ce0-9322551c048a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:50.869787 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d80912-bf73-454d-bf47-01b9fe3fdd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.870996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d80912-bf73-454d-bf47-01b9fe3fdd9b None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.871280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d80912-bf73-454d-bf47-01b9fe3fdd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.871557 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d80912-bf73-454d-bf47-01b9fe3fdd9b None admin] PATH_INFO: `/v3/projects/8458544c021049c587856343e8001429/users/e96b136657864104a51900debd1fbe04/roles/b29c780ef8924837b89e1f06648a9563` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.874124 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7fda5ffc-eacb-408e-8ce0-9322551c048a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:50.881313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:50.882232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:50.883141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:50.883141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:50.883141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:50.883141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:50.885181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 307/611] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:50.894051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d80912-bf73-454d-bf47-01b9fe3fdd9b None admin] RBAC: Authorizing `identity:create_grant(project_id=8458544c021049c587856343e8001429, user_id=e96b136657864104a51900debd1fbe04, role_id=b29c780ef8924837b89e1f06648a9563)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:50.895585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d80912-bf73-454d-bf47-01b9fe3fdd9b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:50.901204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda5ffc-eacb-408e-8ce0-9322551c048a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.913855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 305/612] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:50 2022] PUT /identity/v3/projects/8458544c021049c587856343e8001429/users/e96b136657864104a51900debd1fbe04/roles/b29c780ef8924837b89e1f06648a9563 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:50.949825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538718cf-a50f-4706-8bc1-bdc5fe99e88a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.950535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538718cf-a50f-4706-8bc1-bdc5fe99e88a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.950994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538718cf-a50f-4706-8bc1-bdc5fe99e88a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.952873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 306/613] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:50.961176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9d62-fe7a-4e5c-8a3f-e84c9fd4c4ff None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.961769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9d62-fe7a-4e5c-8a3f-e84c9fd4c4ff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.962257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9d62-fe7a-4e5c-8a3f-e84c9fd4c4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.988215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c8bb9d62-fe7a-4e5c-8a3f-e84c9fd4c4ff None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:50.989734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c8bb9d62-fe7a-4e5c-8a3f-e84c9fd4c4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:51.035682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda5ffc-eacb-408e-8ce0-9322551c048a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:51.100633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88df53-86c9-49c9-accf-75131f3d2ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.101355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88df53-86c9-49c9-accf-75131f3d2ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.101605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88df53-86c9-49c9-accf-75131f3d2ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.101873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88df53-86c9-49c9-accf-75131f3d2ac0 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.102680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88df53-86c9-49c9-accf-75131f3d2ac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:51.102991 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88df53-86c9-49c9-accf-75131f3d2ac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.104580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88df53-86c9-49c9-accf-75131f3d2ac0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.117605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 307/614] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:51.119971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 308/615] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:51.125697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb9d62-fe7a-4e5c-8a3f-e84c9fd4c4ff None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.149520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88df53-86c9-49c9-accf-75131f3d2ac0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.211868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:51.261201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb9d62-fe7a-4e5c-8a3f-e84c9fd4c4ff None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:51.286222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:51.302489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88df53-86c9-49c9-accf-75131f3d2ac0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:51.324625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe641685-3433-4380-a6e0-ee4170ba8669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.325423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe641685-3433-4380-a6e0-ee4170ba8669 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.325703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe641685-3433-4380-a6e0-ee4170ba8669 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.325977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe641685-3433-4380-a6e0-ee4170ba8669 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.338966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe641685-3433-4380-a6e0-ee4170ba8669 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.340441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe641685-3433-4380-a6e0-ee4170ba8669 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.366691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9ef4a-24f8-4798-b678-1fa69e77bba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.367490 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9ef4a-24f8-4798-b678-1fa69e77bba6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.367803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9ef4a-24f8-4798-b678-1fa69e77bba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.368081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9ef4a-24f8-4798-b678-1fa69e77bba6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fe641685-3433-4380-a6e0-ee4170ba8669 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:51.369530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:51.370625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:51.371559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:51.371559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:51.371559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:51.371559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:51.371559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:51.371559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:51.371559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:51.371559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:51.371559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:49:51.371559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:51.371559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 308/616] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:51.380572 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe641685-3433-4380-a6e0-ee4170ba8669 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.383506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9ef4a-24f8-4798-b678-1fa69e77bba6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-69e9ef4a-24f8-4798-b678-1fa69e77bba6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:51.392175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:51.393117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:51.393936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:51.393936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:51.393936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:51.393936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:51.393936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:51.393936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:51.393936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:51.393936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:51.393936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:51.393936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:51.393936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:51.393936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:51.396649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 309/617] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:51.402681 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9ef4a-24f8-4798-b678-1fa69e77bba6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.538973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe641685-3433-4380-a6e0-ee4170ba8669 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:51.579522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42a562-ffdf-4b4c-a367-7bb8a046c9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.580222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42a562-ffdf-4b4c-a367-7bb8a046c9cd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.580507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42a562-ffdf-4b4c-a367-7bb8a046c9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.580785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42a562-ffdf-4b4c-a367-7bb8a046c9cd None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.581776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42a562-ffdf-4b4c-a367-7bb8a046c9cd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:51.582103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42a562-ffdf-4b4c-a367-7bb8a046c9cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.583918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42a562-ffdf-4b4c-a367-7bb8a046c9cd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.602451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9ef4a-24f8-4798-b678-1fa69e77bba6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:51.603508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 309/618] 10.222.0.65 () {56 vars in 1135 bytes} [Thu Jun 23 10:49:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:51.606157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:51.609414 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42a562-ffdf-4b4c-a367-7bb8a046c9cd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.644141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:51.712293 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42a562-ffdf-4b4c-a367-7bb8a046c9cd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:51.731945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933a45f-7373-42f2-af46-6b3e8b2f0cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.733956 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933a45f-7373-42f2-af46-6b3e8b2f0cfb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.735311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933a45f-7373-42f2-af46-6b3e8b2f0cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.735825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933a45f-7373-42f2-af46-6b3e8b2f0cfb None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.736982 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933a45f-7373-42f2-af46-6b3e8b2f0cfb None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:51.738309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933a45f-7373-42f2-af46-6b3e8b2f0cfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.742352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933a45f-7373-42f2-af46-6b3e8b2f0cfb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.757552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 310/619] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:51.761688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933a45f-7373-42f2-af46-6b3e8b2f0cfb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.769972 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453131c-cfe4-4017-93f4-c5b3adc48da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.775627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453131c-cfe4-4017-93f4-c5b3adc48da9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.776125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453131c-cfe4-4017-93f4-c5b3adc48da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.776586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453131c-cfe4-4017-93f4-c5b3adc48da9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.795768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453131c-cfe4-4017-93f4-c5b3adc48da9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.798785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453131c-cfe4-4017-93f4-c5b3adc48da9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b453131c-cfe4-4017-93f4-c5b3adc48da9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:51.823975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:51.825057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:51.825974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:51.825974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:51.825974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:51.825974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:51.828189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 310/620] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:51.836612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453131c-cfe4-4017-93f4-c5b3adc48da9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.911262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933a45f-7373-42f2-af46-6b3e8b2f0cfb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:51.949665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea66723-fb26-4ff6-aef6-5771ed19c559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.950467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea66723-fb26-4ff6-aef6-5771ed19c559 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.950752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea66723-fb26-4ff6-aef6-5771ed19c559 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.951548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea66723-fb26-4ff6-aef6-5771ed19c559 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.964032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea66723-fb26-4ff6-aef6-5771ed19c559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.966076 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea66723-fb26-4ff6-aef6-5771ed19c559 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-cea66723-fb26-4ff6-aef6-5771ed19c559 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:51.985719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:51.986887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:51.988038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:51.988038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:51.988038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:51.988038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:51.988038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 311/621] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:51.992712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea66723-fb26-4ff6-aef6-5771ed19c559 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:52.016934 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:52.102392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea66723-fb26-4ff6-aef6-5771ed19c559 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.141497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f142ccc-cba6-47a7-8828-a9d0819fa6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:52.142171 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f142ccc-cba6-47a7-8828-a9d0819fa6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.142457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f142ccc-cba6-47a7-8828-a9d0819fa6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.142735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f142ccc-cba6-47a7-8828-a9d0819fa6b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.143459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f142ccc-cba6-47a7-8828-a9d0819fa6b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:52.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f142ccc-cba6-47a7-8828-a9d0819fa6b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:52.145324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f142ccc-cba6-47a7-8828-a9d0819fa6b0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:52.153281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:52.164459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 312/622] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:52.165820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f142ccc-cba6-47a7-8828-a9d0819fa6b0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:52.221582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453131c-cfe4-4017-93f4-c5b3adc48da9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.248561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:52.273132 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f142ccc-cba6-47a7-8828-a9d0819fa6b0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.294721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41479707-5bb4-4661-b605-a8aecd1ff1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:52.296298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41479707-5bb4-4661-b605-a8aecd1ff1dc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.297301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41479707-5bb4-4661-b605-a8aecd1ff1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.297731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41479707-5bb4-4661-b605-a8aecd1ff1dc None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.298893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41479707-5bb4-4661-b605-a8aecd1ff1dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:52.299597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41479707-5bb4-4661-b605-a8aecd1ff1dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:52.301519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41479707-5bb4-4661-b605-a8aecd1ff1dc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:52.318318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64775b62-95de-4973-9fc6-c29605ab318c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:52.318948 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 311/623] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:52.319333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775b62-95de-4973-9fc6-c29605ab318c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.319750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775b62-95de-4973-9fc6-c29605ab318c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.320122 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775b62-95de-4973-9fc6-c29605ab318c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.321173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a088dbf-4e2c-45b8-a91f-8e8bd0ee77ee None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.321613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a088dbf-4e2c-45b8-a91f-8e8bd0ee77ee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.322037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a088dbf-4e2c-45b8-a91f-8e8bd0ee77ee None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.329183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 312/624] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:52.330282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64775b62-95de-4973-9fc6-c29605ab318c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:52.330568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a088dbf-4e2c-45b8-a91f-8e8bd0ee77ee None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:52.332127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64775b62-95de-4973-9fc6-c29605ab318c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-64775b62-95de-4973-9fc6-c29605ab318c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:52.346645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:52.347848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:52.349048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:52.349048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:52.349048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:52.349048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:52.349048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:52.349048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:52.349048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:52.349048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:52.349048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:49:52.349048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:52.350849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 313/625] 10.222.0.65 () {56 vars in 1093 bytes} [Thu Jun 23 10:49:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:52.355781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa57622-6b44-4f1d-aa42-831cdb163ac9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.355781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa57622-6b44-4f1d-aa42-831cdb163ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.355781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa57622-6b44-4f1d-aa42-831cdb163ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.379453 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8fa57622-6b44-4f1d-aa42-831cdb163ac9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:52.381911 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8fa57622-6b44-4f1d-aa42-831cdb163ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:52.464373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a088dbf-4e2c-45b8-a91f-8e8bd0ee77ee None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.469417 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 314/626] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:52.470419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa57622-6b44-4f1d-aa42-831cdb163ac9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:52.501621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a2ee1-9c3a-4a82-b114-655d3e7ec3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:52.502763 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a2ee1-9c3a-4a82-b114-655d3e7ec3cd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.503270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a2ee1-9c3a-4a82-b114-655d3e7ec3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.503778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a2ee1-9c3a-4a82-b114-655d3e7ec3cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.511123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a2ee1-9c3a-4a82-b114-655d3e7ec3cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:52.512820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a2ee1-9c3a-4a82-b114-655d3e7ec3cd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-697a2ee1-9c3a-4a82-b114-655d3e7ec3cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:52.527609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:52.528552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:52.529479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:52.529479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:52.529479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:52.529479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:52.529479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:52.529479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:52.529479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:52.529479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:52.536413 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 313/627] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:52.537542 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a2ee1-9c3a-4a82-b114-655d3e7ec3cd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:52.543086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:52.550738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:52.591160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa57622-6b44-4f1d-aa42-831cdb163ac9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.628091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c388f7-cf79-4e1b-af58-9b87ef10ec27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:52.629082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c388f7-cf79-4e1b-af58-9b87ef10ec27 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.629666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c388f7-cf79-4e1b-af58-9b87ef10ec27 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.630143 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c388f7-cf79-4e1b-af58-9b87ef10ec27 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.631195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c388f7-cf79-4e1b-af58-9b87ef10ec27 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:52.631739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c388f7-cf79-4e1b-af58-9b87ef10ec27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:52.633382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c388f7-cf79-4e1b-af58-9b87ef10ec27 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:52.655930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 315/628] 10.222.0.65 () {56 vars in 1133 bytes} [Thu Jun 23 10:49:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:52.657131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c388f7-cf79-4e1b-af58-9b87ef10ec27 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:52.671012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a2ee1-9c3a-4a82-b114-655d3e7ec3cd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.713229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49307ea2-2de7-43e3-86c1-e7ef62445ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:52.714366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49307ea2-2de7-43e3-86c1-e7ef62445ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.714894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49307ea2-2de7-43e3-86c1-e7ef62445ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.715370 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49307ea2-2de7-43e3-86c1-e7ef62445ab6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.726185 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49307ea2-2de7-43e3-86c1-e7ef62445ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-49307ea2-2de7-43e3-86c1-e7ef62445ab6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:52.735043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:52.736026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:52.736913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:52.736913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:52.736913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:52.736913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:52.736913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:52.736913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:52.736913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:52.736913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:52.736913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:52.736913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:52.736913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:52.736913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:52.739644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 314/629] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:52.740738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49307ea2-2de7-43e3-86c1-e7ef62445ab6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:52.774697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c388f7-cf79-4e1b-af58-9b87ef10ec27 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.806916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:52.807229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:52.822555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c757187-a8a6-4fd0-9896-97a4f6942240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:52.823468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c757187-a8a6-4fd0-9896-97a4f6942240 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.823918 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c757187-a8a6-4fd0-9896-97a4f6942240 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.824354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c757187-a8a6-4fd0-9896-97a4f6942240 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.825826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c757187-a8a6-4fd0-9896-97a4f6942240 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:52.826311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c757187-a8a6-4fd0-9896-97a4f6942240 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:52.828602 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c757187-a8a6-4fd0-9896-97a4f6942240 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:52.848845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 316/630] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:52.849937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c757187-a8a6-4fd0-9896-97a4f6942240 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:52.856753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49307ea2-2de7-43e3-86c1-e7ef62445ab6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.903694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d9f62-8c7e-43f2-8661-64f9df924eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:52.904938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d9f62-8c7e-43f2-8661-64f9df924eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.905740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d9f62-8c7e-43f2-8661-64f9df924eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.906161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d9f62-8c7e-43f2-8661-64f9df924eb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.970555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c757187-a8a6-4fd0-9896-97a4f6942240 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.980373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d9f62-8c7e-43f2-8661-64f9df924eb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:52.982203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d9f62-8c7e-43f2-8661-64f9df924eb6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:53.006635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f712ac-8189-48db-88f8-ac10d00a00ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:53.007310 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f712ac-8189-48db-88f8-ac10d00a00ed None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.007596 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f712ac-8189-48db-88f8-ac10d00a00ed None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.007907 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f712ac-8189-48db-88f8-ac10d00a00ed None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.008667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f712ac-8189-48db-88f8-ac10d00a00ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:53.008981 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f712ac-8189-48db-88f8-ac10d00a00ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:53.010354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f712ac-8189-48db-88f8-ac10d00a00ed None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:53.019017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 317/631] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:53.019824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f712ac-8189-48db-88f8-ac10d00a00ed None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:53.053105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:53.069390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:53.102926 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 315/632] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:53.104300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebeb1d-efcf-467c-aaa0-6b5c5bd2aa1f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.104597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebeb1d-efcf-467c-aaa0-6b5c5bd2aa1f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.104880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebeb1d-efcf-467c-aaa0-6b5c5bd2aa1f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.106305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 316/633] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:53.110762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebeb1d-efcf-467c-aaa0-6b5c5bd2aa1f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:53.130783 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f712ac-8189-48db-88f8-ac10d00a00ed None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:53.166277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b8285-3cbb-462d-b6ba-117640c081ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:53.167368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b8285-3cbb-462d-b6ba-117640c081ee None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.167670 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b8285-3cbb-462d-b6ba-117640c081ee None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.167948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b8285-3cbb-462d-b6ba-117640c081ee None admin] PATH_INFO: `/v3/projects/8458544c021049c587856343e8001429/users/d3d8f8acd0484857b451877baf03e895/roles/b29c780ef8924837b89e1f06648a9563` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.184683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b8285-3cbb-462d-b6ba-117640c081ee None admin] RBAC: Authorizing `identity:create_grant(project_id=8458544c021049c587856343e8001429, user_id=d3d8f8acd0484857b451877baf03e895, role_id=b29c780ef8924837b89e1f06648a9563)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:53.187169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b8285-3cbb-462d-b6ba-117640c081ee None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:53.207694 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 318/634] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:53 2022] PUT /identity/v3/projects/8458544c021049c587856343e8001429/users/d3d8f8acd0484857b451877baf03e895/roles/b29c780ef8924837b89e1f06648a9563 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:53.207694 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b8285-3cbb-462d-b6ba-117640c081ee None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:53.232573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebeb1d-efcf-467c-aaa0-6b5c5bd2aa1f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:53.274658 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7c190-7369-403c-a744-36918bbed499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:53.275594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7c190-7369-403c-a744-36918bbed499 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.275998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7c190-7369-403c-a744-36918bbed499 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.276372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7c190-7369-403c-a744-36918bbed499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.289525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7c190-7369-403c-a744-36918bbed499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:53.292641 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7c190-7369-403c-a744-36918bbed499 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-93f7c190-7369-403c-a744-36918bbed499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:53.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:53.317685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:53.318578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:53.318578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:53.318578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:53.318578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:53.318578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 317/635] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:53.320898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d9306-b85a-46b0-b706-4e23090be550 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.321487 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d9306-b85a-46b0-b706-4e23090be550 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.322086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d9306-b85a-46b0-b706-4e23090be550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.341407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:53.345459 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-991d9306-b85a-46b0-b706-4e23090be550 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:53.346785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-991d9306-b85a-46b0-b706-4e23090be550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:53.349218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b8285-3cbb-462d-b6ba-117640c081ee None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:53.361602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:53.394792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48846d79-2b20-46f9-a5d3-c7ef348521eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:53.395760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48846d79-2b20-46f9-a5d3-c7ef348521eb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.396047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48846d79-2b20-46f9-a5d3-c7ef348521eb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.396327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48846d79-2b20-46f9-a5d3-c7ef348521eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.407275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48846d79-2b20-46f9-a5d3-c7ef348521eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:53.408799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48846d79-2b20-46f9-a5d3-c7ef348521eb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-48846d79-2b20-46f9-a5d3-c7ef348521eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:53.426768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:53.427742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:53.428641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:53.428641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:53.428641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:53.428641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:53.428641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 319/636] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:53.428917 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48846d79-2b20-46f9-a5d3-c7ef348521eb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:53.434037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 318/637] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:53.434037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630268e-cada-43d5-b46a-9950fc093194 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.434037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630268e-cada-43d5-b46a-9950fc093194 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.434037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630268e-cada-43d5-b46a-9950fc093194 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.436120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 319/638] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:53.437099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630268e-cada-43d5-b46a-9950fc093194 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:53.539888 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48846d79-2b20-46f9-a5d3-c7ef348521eb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:53.545491 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630268e-cada-43d5-b46a-9950fc093194 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:53.576469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920652f-b4db-4ade-be18-f2c6745a9277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:53.577131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920652f-b4db-4ade-be18-f2c6745a9277 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.577412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920652f-b4db-4ade-be18-f2c6745a9277 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.577688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920652f-b4db-4ade-be18-f2c6745a9277 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.578605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920652f-b4db-4ade-be18-f2c6745a9277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:53.578919 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920652f-b4db-4ade-be18-f2c6745a9277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:53.580476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920652f-b4db-4ade-be18-f2c6745a9277 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:53.581001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd177f6d-11c7-4604-9137-ee3d3b1d13a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:53.581739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd177f6d-11c7-4604-9137-ee3d3b1d13a1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.582093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd177f6d-11c7-4604-9137-ee3d3b1d13a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.582432 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd177f6d-11c7-4604-9137-ee3d3b1d13a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.583227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd177f6d-11c7-4604-9137-ee3d3b1d13a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:53.583605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd177f6d-11c7-4604-9137-ee3d3b1d13a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:53.585060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd177f6d-11c7-4604-9137-ee3d3b1d13a1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:53.596634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 320/639] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:53.600983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb57d6-ec1a-48dc-84c6-ba0d3190fb0a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.600983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb57d6-ec1a-48dc-84c6-ba0d3190fb0a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.600983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb57d6-ec1a-48dc-84c6-ba0d3190fb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.601605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 320/640] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:53.602520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd177f6d-11c7-4604-9137-ee3d3b1d13a1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:53.612254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:53.615687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-91fb57d6-ec1a-48dc-84c6-ba0d3190fb0a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:53.616866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-91fb57d6-ec1a-48dc-84c6-ba0d3190fb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:53.618827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:53.687563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 321/641] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:53.688360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb57d6-ec1a-48dc-84c6-ba0d3190fb0a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:53.718826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd177f6d-11c7-4604-9137-ee3d3b1d13a1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:53.756565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7585e2-9ed3-42a3-99f8-b0d5d1f3a072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:53.757467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7585e2-9ed3-42a3-99f8-b0d5d1f3a072 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.757818 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7585e2-9ed3-42a3-99f8-b0d5d1f3a072 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.758161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7585e2-9ed3-42a3-99f8-b0d5d1f3a072 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.768557 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7585e2-9ed3-42a3-99f8-b0d5d1f3a072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4b7585e2-9ed3-42a3-99f8-b0d5d1f3a072 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:53.776357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:53.777501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:53.778541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:53.778541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:53.778541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:53.778541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:53.778541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:53.778541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:53.778541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:53.778541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:53.778541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:53.778541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:53.778541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:53.778541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:53.780165 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 321/642] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:53.780830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7585e2-9ed3-42a3-99f8-b0d5d1f3a072 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:53.806721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb57d6-ec1a-48dc-84c6-ba0d3190fb0a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:53.842618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175d763-183e-41d4-b811-c9c7a987bd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:53.843638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175d763-183e-41d4-b811-c9c7a987bd76 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.844108 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175d763-183e-41d4-b811-c9c7a987bd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.844475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175d763-183e-41d4-b811-c9c7a987bd76 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.847458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:53.853897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175d763-183e-41d4-b811-c9c7a987bd76 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:53.855429 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175d763-183e-41d4-b811-c9c7a987bd76 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:53.858254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c175d763-183e-41d4-b811-c9c7a987bd76 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:53.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:53.873958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:53.874921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:53.874921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:53.874921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:53.874921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:53.874921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:53.874921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:53.874921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:53.874921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:53.874921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:49:53.874921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:53.874921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 322/643] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:53.874921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175d763-183e-41d4-b811-c9c7a987bd76 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:53.889251 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7585e2-9ed3-42a3-99f8-b0d5d1f3a072 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:53.923818 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6948ffc-8861-410e-8975-1d1fba65e8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:53.924874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6948ffc-8861-410e-8975-1d1fba65e8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.925315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6948ffc-8861-410e-8975-1d1fba65e8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.925605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6948ffc-8861-410e-8975-1d1fba65e8c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.932521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6948ffc-8861-410e-8975-1d1fba65e8c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:53.933962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6948ffc-8861-410e-8975-1d1fba65e8c2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b6948ffc-8861-410e-8975-1d1fba65e8c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:53.948751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:53.949633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:53.950680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:53.950680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:53.950680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:53.950680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:53.950680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:53.950680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:53.950680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:53.950680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:53.950680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 322/644] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:53.950680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6948ffc-8861-410e-8975-1d1fba65e8c2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:53.990461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175d763-183e-41d4-b811-c9c7a987bd76 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.028562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fc3df-5eef-4a84-869c-38c3323a344e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.029892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fc3df-5eef-4a84-869c-38c3323a344e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.030179 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fc3df-5eef-4a84-869c-38c3323a344e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.030465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fc3df-5eef-4a84-869c-38c3323a344e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.041808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258fc3df-5eef-4a84-869c-38c3323a344e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-258fc3df-5eef-4a84-869c-38c3323a344e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:54.050026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:54.050954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:54.051824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 323/645] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:54.053112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fc3df-5eef-4a84-869c-38c3323a344e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.061807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6948ffc-8861-410e-8975-1d1fba65e8c2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.091146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.102642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76629-9328-458f-8b36-1e08a07307a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.103307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76629-9328-458f-8b36-1e08a07307a7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.103585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76629-9328-458f-8b36-1e08a07307a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.103931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76629-9328-458f-8b36-1e08a07307a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.104885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76629-9328-458f-8b36-1e08a07307a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:54.105199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76629-9328-458f-8b36-1e08a07307a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.106691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76629-9328-458f-8b36-1e08a07307a7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.116627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 323/646] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:54.121264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.123018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76629-9328-458f-8b36-1e08a07307a7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.164882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fc3df-5eef-4a84-869c-38c3323a344e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.199355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f76f1-5835-40de-8b97-a5a6e07d3811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.200361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f76f1-5835-40de-8b97-a5a6e07d3811 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.200714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f76f1-5835-40de-8b97-a5a6e07d3811 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.201611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f76f1-5835-40de-8b97-a5a6e07d3811 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.202516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f76f1-5835-40de-8b97-a5a6e07d3811 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:54.202893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f76f1-5835-40de-8b97-a5a6e07d3811 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.204433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f76f1-5835-40de-8b97-a5a6e07d3811 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.222627 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 324/647] 10.222.0.65 () {56 vars in 1137 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:54.223670 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f76f1-5835-40de-8b97-a5a6e07d3811 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.235623 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76629-9328-458f-8b36-1e08a07307a7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.272279 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec66b41-143f-4080-b1f6-5ac20d6984ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.272877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec66b41-143f-4080-b1f6-5ac20d6984ec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.273162 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec66b41-143f-4080-b1f6-5ac20d6984ec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.273689 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec66b41-143f-4080-b1f6-5ac20d6984ec None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.274208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec66b41-143f-4080-b1f6-5ac20d6984ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:54.274518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec66b41-143f-4080-b1f6-5ac20d6984ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.275956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec66b41-143f-4080-b1f6-5ac20d6984ec None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.293854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 324/648] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:54.293854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec66b41-143f-4080-b1f6-5ac20d6984ec None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.332476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f76f1-5835-40de-8b97-a5a6e07d3811 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.343035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.352146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.372120 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2da2ea-8c09-4394-8035-87ad9af21616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.372793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2da2ea-8c09-4394-8035-87ad9af21616 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.373100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2da2ea-8c09-4394-8035-87ad9af21616 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.373374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2da2ea-8c09-4394-8035-87ad9af21616 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.374119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2da2ea-8c09-4394-8035-87ad9af21616 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:54.374425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2da2ea-8c09-4394-8035-87ad9af21616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.376257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2da2ea-8c09-4394-8035-87ad9af21616 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.389851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 325/649] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:54.389851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2da2ea-8c09-4394-8035-87ad9af21616 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.406666 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec66b41-143f-4080-b1f6-5ac20d6984ec None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.441203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dd811-8bc8-437b-9fea-1c50b347d72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.441988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dd811-8bc8-437b-9fea-1c50b347d72f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.442269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dd811-8bc8-437b-9fea-1c50b347d72f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.442563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dd811-8bc8-437b-9fea-1c50b347d72f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.454114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9dd811-8bc8-437b-9fea-1c50b347d72f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.455639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9dd811-8bc8-437b-9fea-1c50b347d72f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2d9dd811-8bc8-437b-9fea-1c50b347d72f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:54.474852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:54.475789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:54.476717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:54.476717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:54.476717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:54.476717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:54.476717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 325/650] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:54.476998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dd811-8bc8-437b-9fea-1c50b347d72f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.491559 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2da2ea-8c09-4394-8035-87ad9af21616 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.527000 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71667753-ed62-4d4b-a7c9-36ac93eb33fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.527647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71667753-ed62-4d4b-a7c9-36ac93eb33fc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.527951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71667753-ed62-4d4b-a7c9-36ac93eb33fc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.528224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71667753-ed62-4d4b-a7c9-36ac93eb33fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.575068 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.581372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.584883 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dd811-8bc8-437b-9fea-1c50b347d72f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.598466 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71667753-ed62-4d4b-a7c9-36ac93eb33fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.599957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71667753-ed62-4d4b-a7c9-36ac93eb33fc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.620459 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909c154-c262-418b-b51f-ff53e69ba6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.621117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909c154-c262-418b-b51f-ff53e69ba6cc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.621407 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909c154-c262-418b-b51f-ff53e69ba6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.621684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909c154-c262-418b-b51f-ff53e69ba6cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.622509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909c154-c262-418b-b51f-ff53e69ba6cc None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=b01d875111bd43698bcf99d30f7a7f9a) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:54.623326 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909c154-c262-418b-b51f-ff53e69ba6cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.624779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909c154-c262-418b-b51f-ff53e69ba6cc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.630626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 326/651] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/role_assignments?user.id=b01d875111bd43698bcf99d30f7a7f9a&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:54.631406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909c154-c262-418b-b51f-ff53e69ba6cc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.728505 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 326/652] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:54.729445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71667753-ed62-4d4b-a7c9-36ac93eb33fc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.787285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909c154-c262-418b-b51f-ff53e69ba6cc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.825384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.830660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1067ee-6301-4525-9112-4a49b9bf72b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.831575 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1067ee-6301-4525-9112-4a49b9bf72b9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.831885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1067ee-6301-4525-9112-4a49b9bf72b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.832652 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1067ee-6301-4525-9112-4a49b9bf72b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.846974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1067ee-6301-4525-9112-4a49b9bf72b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.848481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1067ee-6301-4525-9112-4a49b9bf72b9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.858659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71667753-ed62-4d4b-a7c9-36ac93eb33fc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ae1067ee-6301-4525-9112-4a49b9bf72b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:54.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:54.870330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:54.871224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:54.871224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:54.871224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:54.871224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:54.874025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 327/653] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:54.875110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1067ee-6301-4525-9112-4a49b9bf72b9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.898226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.902560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1309060-12f1-45ba-9543-b4a852c97308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.903249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1309060-12f1-45ba-9543-b4a852c97308 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.903534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1309060-12f1-45ba-9543-b4a852c97308 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.903861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1309060-12f1-45ba-9543-b4a852c97308 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.905105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1309060-12f1-45ba-9543-b4a852c97308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:54.905416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1309060-12f1-45ba-9543-b4a852c97308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.907302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1309060-12f1-45ba-9543-b4a852c97308 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.922648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 327/654] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:54.923974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1309060-12f1-45ba-9543-b4a852c97308 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.023454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1067ee-6301-4525-9112-4a49b9bf72b9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.044278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1309060-12f1-45ba-9543-b4a852c97308 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.067171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de511e7-d0da-411e-8c53-871c900458cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.068024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de511e7-d0da-411e-8c53-871c900458cc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.068305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de511e7-d0da-411e-8c53-871c900458cc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.068581 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de511e7-d0da-411e-8c53-871c900458cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.083169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de511e7-d0da-411e-8c53-871c900458cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.084423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec734a-7b67-4429-8f23-a32c9e5cf37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.084739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de511e7-d0da-411e-8c53-871c900458cc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.085589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec734a-7b67-4429-8f23-a32c9e5cf37b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.086073 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec734a-7b67-4429-8f23-a32c9e5cf37b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.086667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec734a-7b67-4429-8f23-a32c9e5cf37b None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.087727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec734a-7b67-4429-8f23-a32c9e5cf37b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:55.088238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec734a-7b67-4429-8f23-a32c9e5cf37b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.089791 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec734a-7b67-4429-8f23-a32c9e5cf37b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.101822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2de511e7-d0da-411e-8c53-871c900458cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:55.112126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:55.113042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:55.113937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:55.113937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:55.113937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:55.113937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:55.117912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 328/656] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:55.118133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 328/656] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:55.118850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de511e7-d0da-411e-8c53-871c900458cc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.120404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec734a-7b67-4429-8f23-a32c9e5cf37b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.190947 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.233270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de511e7-d0da-411e-8c53-871c900458cc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.253564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec734a-7b67-4429-8f23-a32c9e5cf37b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.269965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2034c4-0384-4adc-9a8a-8df7e2d5bbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.270680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2034c4-0384-4adc-9a8a-8df7e2d5bbef None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.270965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2034c4-0384-4adc-9a8a-8df7e2d5bbef None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.271243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2034c4-0384-4adc-9a8a-8df7e2d5bbef None admin] PATH_INFO: `/v3/users/glance` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.279955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2034c4-0384-4adc-9a8a-8df7e2d5bbef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.281714 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2034c4-0384-4adc-9a8a-8df7e2d5bbef None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.291691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85033e59-cc7e-4cec-a016-4aecee3cc5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.292360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85033e59-cc7e-4cec-a016-4aecee3cc5bb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.292641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85033e59-cc7e-4cec-a016-4aecee3cc5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.292914 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85033e59-cc7e-4cec-a016-4aecee3cc5bb None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.293632 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85033e59-cc7e-4cec-a016-4aecee3cc5bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:55.293941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85033e59-cc7e-4cec-a016-4aecee3cc5bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.295389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85033e59-cc7e-4cec-a016-4aecee3cc5bb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6e2034c4-0384-4adc-9a8a-8df7e2d5bbef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:55.296704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:55.297642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:55.298579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:55.298579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:55.298579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:55.298579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:55.298579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:55.298579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:55.298579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:55.298579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:55.298579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 10:49:55.298579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:55.298579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 329/657] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:55.299169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2034c4-0384-4adc-9a8a-8df7e2d5bbef None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.312863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 329/658] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:55.313987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb2bea-8d68-47a4-9f42-d82630e2f2f8 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.314274 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb2bea-8d68-47a4-9f42-d82630e2f2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.314550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb2bea-8d68-47a4-9f42-d82630e2f2f8 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.315796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 330/659] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:55.319757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb2bea-8d68-47a4-9f42-d82630e2f2f8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.380867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.421972 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2034c4-0384-4adc-9a8a-8df7e2d5bbef None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.424186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb2bea-8d68-47a4-9f42-d82630e2f2f8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.449346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.457997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaea14f-ea8b-4b1e-b0c0-c058d7a81da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.458264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c3642-723c-4285-9b83-61051f53dd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.458800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaea14f-ea8b-4b1e-b0c0-c058d7a81da8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.458910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c3642-723c-4285-9b83-61051f53dd55 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.459078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaea14f-ea8b-4b1e-b0c0-c058d7a81da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.459174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c3642-723c-4285-9b83-61051f53dd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.459356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaea14f-ea8b-4b1e-b0c0-c058d7a81da8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.459451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c3642-723c-4285-9b83-61051f53dd55 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.460299 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c3642-723c-4285-9b83-61051f53dd55 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:55.460615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c3642-723c-4285-9b83-61051f53dd55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.462151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c3642-723c-4285-9b83-61051f53dd55 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.467892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaea14f-ea8b-4b1e-b0c0-c058d7a81da8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.469330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaea14f-ea8b-4b1e-b0c0-c058d7a81da8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3aaea14f-ea8b-4b1e-b0c0-c058d7a81da8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:55.484307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:55.485275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:55.490913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:55.490913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:55.490913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:55.490913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:55.490913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:55.490913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:55.490913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:55.490913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:55.490913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 10:49:55.490913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:55.491455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 331/660] 10.222.0.65 () {56 vars in 1137 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:55.491540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 330/661] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:55.491611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c3642-723c-4285-9b83-61051f53dd55 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.491717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69e282-73ce-46b8-88b1-e0640670142b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.491717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69e282-73ce-46b8-88b1-e0640670142b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.491717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69e282-73ce-46b8-88b1-e0640670142b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.506101 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6a69e282-73ce-46b8-88b1-e0640670142b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:55.507294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a69e282-73ce-46b8-88b1-e0640670142b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:55.579287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 331/662] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:55.580458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69e282-73ce-46b8-88b1-e0640670142b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.604074 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c3642-723c-4285-9b83-61051f53dd55 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.621585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.642147 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3809f32-5805-4651-bf99-024cb868295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.642927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3809f32-5805-4651-bf99-024cb868295c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.643247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3809f32-5805-4651-bf99-024cb868295c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.643487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3809f32-5805-4651-bf99-024cb868295c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.650260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3809f32-5805-4651-bf99-024cb868295c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.652318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3809f32-5805-4651-bf99-024cb868295c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a3809f32-5805-4651-bf99-024cb868295c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:55.666031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:55.667011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:55.668325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:55.668325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:55.668325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:55.668325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:55.668325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:55.668325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:55.668325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:55.668325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:55.668325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 332/663] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:55.668325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3809f32-5805-4651-bf99-024cb868295c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.688495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69e282-73ce-46b8-88b1-e0640670142b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.706129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.725777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982a0f2-64bf-4f35-9b16-811d3120b690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.726426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982a0f2-64bf-4f35-9b16-811d3120b690 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.726707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982a0f2-64bf-4f35-9b16-811d3120b690 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.726984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982a0f2-64bf-4f35-9b16-811d3120b690 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.773502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3809f32-5805-4651-bf99-024cb868295c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.797481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982a0f2-64bf-4f35-9b16-811d3120b690 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.798948 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982a0f2-64bf-4f35-9b16-811d3120b690 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.808613 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df9e51-3649-47a3-b9ae-b028f4350ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.809541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df9e51-3649-47a3-b9ae-b028f4350ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.810081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df9e51-3649-47a3-b9ae-b028f4350ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.810563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df9e51-3649-47a3-b9ae-b028f4350ee5 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.811556 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df9e51-3649-47a3-b9ae-b028f4350ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:55.811891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df9e51-3649-47a3-b9ae-b028f4350ee5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.813309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df9e51-3649-47a3-b9ae-b028f4350ee5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.831705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 333/664] 10.222.0.65 () {56 vars in 1139 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:55.832438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df9e51-3649-47a3-b9ae-b028f4350ee5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.872890 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.924921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 332/665] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:55.924921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982a0f2-64bf-4f35-9b16-811d3120b690 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.941832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.943347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df9e51-3649-47a3-b9ae-b028f4350ee5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.980112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a25b3-676d-4d8c-a5b6-082a1cb095fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.980909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a25b3-676d-4d8c-a5b6-082a1cb095fb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.981191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a25b3-676d-4d8c-a5b6-082a1cb095fb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.981467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a25b3-676d-4d8c-a5b6-082a1cb095fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.992889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a25b3-676d-4d8c-a5b6-082a1cb095fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7d2a25b3-676d-4d8c-a5b6-082a1cb095fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:56.001236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:56.002173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:56.003857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 334/666] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:56.004557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a25b3-676d-4d8c-a5b6-082a1cb095fb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.031938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982a0f2-64bf-4f35-9b16-811d3120b690 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.071879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cea153-9040-4f42-978d-b2bb4890856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.072538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cea153-9040-4f42-978d-b2bb4890856c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.072831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cea153-9040-4f42-978d-b2bb4890856c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.073092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cea153-9040-4f42-978d-b2bb4890856c None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.073866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cea153-9040-4f42-978d-b2bb4890856c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:56.074181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cea153-9040-4f42-978d-b2bb4890856c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.075594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cea153-9040-4f42-978d-b2bb4890856c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.092033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 333/667] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:56.093127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cea153-9040-4f42-978d-b2bb4890856c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.101468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.108834 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a25b3-676d-4d8c-a5b6-082a1cb095fb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.144925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48a90d-9e0b-4c2f-b5ed-1504b3c2b1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.145667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48a90d-9e0b-4c2f-b5ed-1504b3c2b1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.145954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48a90d-9e0b-4c2f-b5ed-1504b3c2b1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.146438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48a90d-9e0b-4c2f-b5ed-1504b3c2b1f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.179194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.202734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cea153-9040-4f42-978d-b2bb4890856c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.217508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48a90d-9e0b-4c2f-b5ed-1504b3c2b1f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.218967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48a90d-9e0b-4c2f-b5ed-1504b3c2b1f4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.237816 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc4068-b281-426b-bf87-1a3263641627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.238675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc4068-b281-426b-bf87-1a3263641627 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.238959 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc4068-b281-426b-bf87-1a3263641627 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.239235 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc4068-b281-426b-bf87-1a3263641627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.250876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc4068-b281-426b-bf87-1a3263641627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.252350 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc4068-b281-426b-bf87-1a3263641627 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ecbc4068-b281-426b-bf87-1a3263641627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:56.270250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:56.271144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:56.272078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:56.272078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:56.272078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:56.272078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:56.272078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 334/668] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:56.272367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc4068-b281-426b-bf87-1a3263641627 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.318299 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.327819 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 335/669] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:56.328855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48a90d-9e0b-4c2f-b5ed-1504b3c2b1f4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.386062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc4068-b281-426b-bf87-1a3263641627 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.415930 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.422977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fbe82-2674-4ef5-a9f2-bb08852a7db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.423694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fbe82-2674-4ef5-a9f2-bb08852a7db9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.423982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fbe82-2674-4ef5-a9f2-bb08852a7db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.424265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fbe82-2674-4ef5-a9f2-bb08852a7db9 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.425036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fbe82-2674-4ef5-a9f2-bb08852a7db9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:56.425443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fbe82-2674-4ef5-a9f2-bb08852a7db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.426869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fbe82-2674-4ef5-a9f2-bb08852a7db9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.437409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48a90d-9e0b-4c2f-b5ed-1504b3c2b1f4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.440894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 335/670] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:56.441559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fbe82-2674-4ef5-a9f2-bb08852a7db9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.472343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d1891-ec41-409c-8dba-54877ec1e67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.473016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d1891-ec41-409c-8dba-54877ec1e67e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.473348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d1891-ec41-409c-8dba-54877ec1e67e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.473643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d1891-ec41-409c-8dba-54877ec1e67e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.474470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d1891-ec41-409c-8dba-54877ec1e67e None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=3663612647934677b83fb42aaf06d6b8) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:56.474786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d1891-ec41-409c-8dba-54877ec1e67e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.476262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d1891-ec41-409c-8dba-54877ec1e67e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.482728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 336/671] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/role_assignments?user.id=3663612647934677b83fb42aaf06d6b8&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:56.483816 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d1891-ec41-409c-8dba-54877ec1e67e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.571036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.572119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fbe82-2674-4ef5-a9f2-bb08852a7db9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.614908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448079e-2915-4e40-96d3-c5f446e26bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.615957 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448079e-2915-4e40-96d3-c5f446e26bac None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.616461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448079e-2915-4e40-96d3-c5f446e26bac None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.616927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448079e-2915-4e40-96d3-c5f446e26bac None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.617873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448079e-2915-4e40-96d3-c5f446e26bac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:56.618385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448079e-2915-4e40-96d3-c5f446e26bac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.620021 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448079e-2915-4e40-96d3-c5f446e26bac None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.628653 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d1891-ec41-409c-8dba-54877ec1e67e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.635037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 336/672] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:56.636364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448079e-2915-4e40-96d3-c5f446e26bac None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.676136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c2119-27fb-4a6a-b179-3c7e192ca29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.677132 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2119-27fb-4a6a-b179-3c7e192ca29b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.677251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2119-27fb-4a6a-b179-3c7e192ca29b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.677481 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2119-27fb-4a6a-b179-3c7e192ca29b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.693088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c2119-27fb-4a6a-b179-3c7e192ca29b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.694554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c2119-27fb-4a6a-b179-3c7e192ca29b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.707368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3e4c2119-27fb-4a6a-b179-3c7e192ca29b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:56.714636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:56.715936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:56.717530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:56.717530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:56.717530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:56.717530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:56.717530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 337/673] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:56.717530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c2119-27fb-4a6a-b179-3c7e192ca29b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.771527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448079e-2915-4e40-96d3-c5f446e26bac None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.806910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f40a9-e92c-42d6-bdca-a729eba6cb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.807736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f40a9-e92c-42d6-bdca-a729eba6cb79 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.808024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f40a9-e92c-42d6-bdca-a729eba6cb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.808304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f40a9-e92c-42d6-bdca-a729eba6cb79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.819844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f40a9-e92c-42d6-bdca-a729eba6cb79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.821552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f40a9-e92c-42d6-bdca-a729eba6cb79 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.835995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c2119-27fb-4a6a-b179-3c7e192ca29b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a37f40a9-e92c-42d6-bdca-a729eba6cb79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:56.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:56.845403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:56.846481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:56.846481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:56.846481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:56.846481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:56.847068 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.847989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 337/674] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:56.848768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f40a9-e92c-42d6-bdca-a729eba6cb79 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.878396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626a938-6c6f-495b-82e7-ee749e7af389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.879197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626a938-6c6f-495b-82e7-ee749e7af389 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.879480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626a938-6c6f-495b-82e7-ee749e7af389 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.879796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626a938-6c6f-495b-82e7-ee749e7af389 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.887001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626a938-6c6f-495b-82e7-ee749e7af389 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.888468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626a938-6c6f-495b-82e7-ee749e7af389 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d626a938-6c6f-495b-82e7-ee749e7af389 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:56.903213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:56.904213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:56.905159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:56.905159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:56.905159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:56.905159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:56.905159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:56.905159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:56.905159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:56.905159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:56.909196 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 338/675] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:56.910395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35e552-c0ef-4344-b230-60201859a068 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.910691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35e552-c0ef-4344-b230-60201859a068 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.910970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35e552-c0ef-4344-b230-60201859a068 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.916746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 339/676] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:56.916746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35e552-c0ef-4344-b230-60201859a068 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.962252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f40a9-e92c-42d6-bdca-a729eba6cb79 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.964861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.003105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e73002-d63a-483a-8b99-e0e97c8d526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.003803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e73002-d63a-483a-8b99-e0e97c8d526f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.004090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e73002-d63a-483a-8b99-e0e97c8d526f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.004370 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e73002-d63a-483a-8b99-e0e97c8d526f None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.005230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e73002-d63a-483a-8b99-e0e97c8d526f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.005541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e73002-d63a-483a-8b99-e0e97c8d526f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.006911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e73002-d63a-483a-8b99-e0e97c8d526f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.019619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 338/677] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.021913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e73002-d63a-483a-8b99-e0e97c8d526f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.025313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35e552-c0ef-4344-b230-60201859a068 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.060013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a07d18-e8cd-433d-b7e7-2655aab33d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.061248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a07d18-e8cd-433d-b7e7-2655aab33d32 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.061530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a07d18-e8cd-433d-b7e7-2655aab33d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.061803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a07d18-e8cd-433d-b7e7-2655aab33d32 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.062515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a07d18-e8cd-433d-b7e7-2655aab33d32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.062820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a07d18-e8cd-433d-b7e7-2655aab33d32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.064212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a07d18-e8cd-433d-b7e7-2655aab33d32 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.080654 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 340/678] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.080654 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1f6b-8610-4a41-8c0d-ae32f869cc23 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.080654 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1f6b-8610-4a41-8c0d-ae32f869cc23 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.080654 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1f6b-8610-4a41-8c0d-ae32f869cc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.102908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-80dc1f6b-8610-4a41-8c0d-ae32f869cc23 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:57.104131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-80dc1f6b-8610-4a41-8c0d-ae32f869cc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:57.131133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.132195 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e73002-d63a-483a-8b99-e0e97c8d526f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.167278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b02f9-42a3-4546-89c8-e61d92382527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.168029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b02f9-42a3-4546-89c8-e61d92382527 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.168311 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b02f9-42a3-4546-89c8-e61d92382527 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.168586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b02f9-42a3-4546-89c8-e61d92382527 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.169374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b02f9-42a3-4546-89c8-e61d92382527 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.170215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b02f9-42a3-4546-89c8-e61d92382527 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.171882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b02f9-42a3-4546-89c8-e61d92382527 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.183842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 341/679] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:57.183842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc1f6b-8610-4a41-8c0d-ae32f869cc23 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.195503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 339/680] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.196629 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b02f9-42a3-4546-89c8-e61d92382527 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.227723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.291069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc1f6b-8610-4a41-8c0d-ae32f869cc23 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.309337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b02f9-42a3-4546-89c8-e61d92382527 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.329051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29116ba5-2be1-427b-a9cd-7f252c3af4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.329851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29116ba5-2be1-427b-a9cd-7f252c3af4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.330135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29116ba5-2be1-427b-a9cd-7f252c3af4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.330409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29116ba5-2be1-427b-a9cd-7f252c3af4b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.337033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29116ba5-2be1-427b-a9cd-7f252c3af4b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29116ba5-2be1-427b-a9cd-7f252c3af4b7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.345384 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f0f18-06a7-4c48-b9b7-1f97aa9c459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.346181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f0f18-06a7-4c48-b9b7-1f97aa9c459b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.346468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f0f18-06a7-4c48-b9b7-1f97aa9c459b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.346747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f0f18-06a7-4c48-b9b7-1f97aa9c459b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.354137 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f0f18-06a7-4c48-b9b7-1f97aa9c459b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.355564 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f0f18-06a7-4c48-b9b7-1f97aa9c459b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-29116ba5-2be1-427b-a9cd-7f252c3af4b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:57.356002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:57.356861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:57.358558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:57.358558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:57.358558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:57.358558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:57.358558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:57.358558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:57.358558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:57.358558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:57.365504 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 342/681] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:57.366363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29116ba5-2be1-427b-a9cd-7f252c3af4b7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cb1f0f18-06a7-4c48-b9b7-1f97aa9c459b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:57.374209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:57.375401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:57.382903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:57.382903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:57.382903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:57.382903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:57.382903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:57.382903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:57.382903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:57.382903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:57.382903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:49:57.382903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:57.382903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 340/682] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:57.382903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f0f18-06a7-4c48-b9b7-1f97aa9c459b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.398618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.465068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.485087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f0f18-06a7-4c48-b9b7-1f97aa9c459b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.485527 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29116ba5-2be1-427b-a9cd-7f252c3af4b7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.520600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9e1fb-010b-4109-8086-7f1461040cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.521396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9e1fb-010b-4109-8086-7f1461040cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.521687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9e1fb-010b-4109-8086-7f1461040cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.521962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9e1fb-010b-4109-8086-7f1461040cc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.522969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a2a73-a1de-4c53-b8bd-9780ad934c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.523715 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a2a73-a1de-4c53-b8bd-9780ad934c62 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.523940 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a2a73-a1de-4c53-b8bd-9780ad934c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.524205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a2a73-a1de-4c53-b8bd-9780ad934c62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.525131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a2a73-a1de-4c53-b8bd-9780ad934c62 None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=5af6cc8c56db48beae2550954446c119) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.525339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a2a73-a1de-4c53-b8bd-9780ad934c62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.526756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a2a73-a1de-4c53-b8bd-9780ad934c62 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.539688 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 341/683] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/role_assignments?user.id=5af6cc8c56db48beae2550954446c119&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.539913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9e1fb-010b-4109-8086-7f1461040cc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:57.540018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a2a73-a1de-4c53-b8bd-9780ad934c62 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-bde9e1fb-010b-4109-8086-7f1461040cc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:57.541948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:57.542864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 343/684] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:57.544239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9e1fb-010b-4109-8086-7f1461040cc4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.672042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.677546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9e1fb-010b-4109-8086-7f1461040cc4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.683396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a2a73-a1de-4c53-b8bd-9780ad934c62 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.725128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce12f99-c699-4102-9e77-8096bdfd2024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.726099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce12f99-c699-4102-9e77-8096bdfd2024 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.726573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce12f99-c699-4102-9e77-8096bdfd2024 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.727125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce12f99-c699-4102-9e77-8096bdfd2024 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.728188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce12f99-c699-4102-9e77-8096bdfd2024 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.728667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce12f99-c699-4102-9e77-8096bdfd2024 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.730409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce12f99-c699-4102-9e77-8096bdfd2024 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.733132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded91d3-cd4e-4351-89a1-e8b82ce5e6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.734760 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded91d3-cd4e-4351-89a1-e8b82ce5e6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.735050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded91d3-cd4e-4351-89a1-e8b82ce5e6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.735332 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded91d3-cd4e-4351-89a1-e8b82ce5e6c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.736841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded91d3-cd4e-4351-89a1-e8b82ce5e6c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.736841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded91d3-cd4e-4351-89a1-e8b82ce5e6c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.737937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded91d3-cd4e-4351-89a1-e8b82ce5e6c0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.753241 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 342/685] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.753918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded91d3-cd4e-4351-89a1-e8b82ce5e6c0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.754686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 344/686] 10.222.0.65 () {56 vars in 1135 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.761061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce12f99-c699-4102-9e77-8096bdfd2024 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.769257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.875200 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded91d3-cd4e-4351-89a1-e8b82ce5e6c0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.878547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce12f99-c699-4102-9e77-8096bdfd2024 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.911065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5462b-9c42-44ae-a2c4-1b00a5a4d69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.911854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5462b-9c42-44ae-a2c4-1b00a5a4d69f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.912144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5462b-9c42-44ae-a2c4-1b00a5a4d69f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.912426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5462b-9c42-44ae-a2c4-1b00a5a4d69f None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.913393 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5462b-9c42-44ae-a2c4-1b00a5a4d69f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.913773 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5462b-9c42-44ae-a2c4-1b00a5a4d69f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.914856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69944f8f-464e-446d-a4da-50924fc2922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.915264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5462b-9c42-44ae-a2c4-1b00a5a4d69f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.915870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69944f8f-464e-446d-a4da-50924fc2922a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.916301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69944f8f-464e-446d-a4da-50924fc2922a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.916725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69944f8f-464e-446d-a4da-50924fc2922a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.917726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69944f8f-464e-446d-a4da-50924fc2922a None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=e96b136657864104a51900debd1fbe04) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.918190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69944f8f-464e-446d-a4da-50924fc2922a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.920315 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69944f8f-464e-446d-a4da-50924fc2922a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.924638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 343/687] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.925409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5462b-9c42-44ae-a2c4-1b00a5a4d69f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.931786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 345/688] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/role_assignments?user.id=e96b136657864104a51900debd1fbe04&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.932920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69944f8f-464e-446d-a4da-50924fc2922a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.942540 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.016026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.041891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5462b-9c42-44ae-a2c4-1b00a5a4d69f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.080470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59924cb9-f622-419a-8008-1d47790b0818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.081145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59924cb9-f622-419a-8008-1d47790b0818 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.081431 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59924cb9-f622-419a-8008-1d47790b0818 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.081708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59924cb9-f622-419a-8008-1d47790b0818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.139693 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69944f8f-464e-446d-a4da-50924fc2922a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.157331 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59924cb9-f622-419a-8008-1d47790b0818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.158964 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59924cb9-f622-419a-8008-1d47790b0818 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.186696 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20346f-8950-4516-8317-c5a3a1b582da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.188262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20346f-8950-4516-8317-c5a3a1b582da None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.188758 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20346f-8950-4516-8317-c5a3a1b582da None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.189254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20346f-8950-4516-8317-c5a3a1b582da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.214726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20346f-8950-4516-8317-c5a3a1b582da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.217383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20346f-8950-4516-8317-c5a3a1b582da None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5a20346f-8950-4516-8317-c5a3a1b582da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:58.238657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:58.239606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:58.240544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:58.240544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:58.240544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:58.240544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:58.240544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 346/689] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:58.249196 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20346f-8950-4516-8317-c5a3a1b582da None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.255439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.292411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 344/690] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:58.298384 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59924cb9-f622-419a-8008-1d47790b0818 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.343702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.408538 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20346f-8950-4516-8317-c5a3a1b582da None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.423693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59924cb9-f622-419a-8008-1d47790b0818 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.459761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef157aa-5ed4-44fb-99ef-6266ef306273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.460681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef157aa-5ed4-44fb-99ef-6266ef306273 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.461475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef157aa-5ed4-44fb-99ef-6266ef306273 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.461973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef157aa-5ed4-44fb-99ef-6266ef306273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.474897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef157aa-5ed4-44fb-99ef-6266ef306273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.476671 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef157aa-5ed4-44fb-99ef-6266ef306273 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cef157aa-5ed4-44fb-99ef-6266ef306273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:58.496859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:58.497836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:58.498831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:58.498831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:58.498831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:58.498831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:58.498831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 345/691] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:58.499882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d8d69-bafe-461d-bc38-0bb4706ebb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.500755 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d8d69-bafe-461d-bc38-0bb4706ebb56 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.501213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d8d69-bafe-461d-bc38-0bb4706ebb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.501655 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d8d69-bafe-461d-bc38-0bb4706ebb56 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.502554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d8d69-bafe-461d-bc38-0bb4706ebb56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:58.503292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d8d69-bafe-461d-bc38-0bb4706ebb56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.503924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef157aa-5ed4-44fb-99ef-6266ef306273 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.505142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d8d69-bafe-461d-bc38-0bb4706ebb56 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.531981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.542027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 347/692] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:58.550081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d8d69-bafe-461d-bc38-0bb4706ebb56 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.624566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef157aa-5ed4-44fb-99ef-6266ef306273 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.667327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de1ef9-e844-4097-8897-e5f4375452a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.668565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de1ef9-e844-4097-8897-e5f4375452a6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.668851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de1ef9-e844-4097-8897-e5f4375452a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.669127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de1ef9-e844-4097-8897-e5f4375452a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.669886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de1ef9-e844-4097-8897-e5f4375452a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:58.670196 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de1ef9-e844-4097-8897-e5f4375452a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.671759 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de1ef9-e844-4097-8897-e5f4375452a6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.685303 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 346/693] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:58.690689 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de1ef9-e844-4097-8897-e5f4375452a6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.761486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d8d69-bafe-461d-bc38-0bb4706ebb56 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.779946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.793023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.804335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5f820-0dc7-4edc-8cef-208eaef04d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.805463 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5f820-0dc7-4edc-8cef-208eaef04d2a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.805937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5f820-0dc7-4edc-8cef-208eaef04d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.806419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5f820-0dc7-4edc-8cef-208eaef04d2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.815406 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5f820-0dc7-4edc-8cef-208eaef04d2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.816974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de1ef9-e844-4097-8897-e5f4375452a6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.817380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5f820-0dc7-4edc-8cef-208eaef04d2a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a8c5f820-0dc7-4edc-8cef-208eaef04d2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:58.834522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:58.835814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:58.836897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:58.836897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:58.836897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:58.836897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:58.836897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:58.836897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:58.836897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:58.836897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:58.836897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:49:58.836897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:49:58.839160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 348/694] 10.222.0.65 () {56 vars in 1093 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:58.840676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a70060-44b2-4f5b-a6a3-277ef1c4cd86 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.841151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a70060-44b2-4f5b-a6a3-277ef1c4cd86 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.842169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a70060-44b2-4f5b-a6a3-277ef1c4cd86 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.843918 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 349/695] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:58.846557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a70060-44b2-4f5b-a6a3-277ef1c4cd86 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.863881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465455e-c76e-452e-815e-824f1325597b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.864745 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465455e-c76e-452e-815e-824f1325597b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.865033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465455e-c76e-452e-815e-824f1325597b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.865473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465455e-c76e-452e-815e-824f1325597b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.876132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465455e-c76e-452e-815e-824f1325597b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.879685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465455e-c76e-452e-815e-824f1325597b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d465455e-c76e-452e-815e-824f1325597b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:58.897891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:58.898801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:58.905218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:58.905218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:58.905218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:58.905218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:58.905218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:58.905218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:58.905218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:58.905218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:58.905218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 347/696] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:58.905737 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396b887-9e2f-4306-b03e-5819ce825f7a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.905737 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396b887-9e2f-4306-b03e-5819ce825f7a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.906145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396b887-9e2f-4306-b03e-5819ce825f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.935460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4396b887-9e2f-4306-b03e-5819ce825f7a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:58.936959 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4396b887-9e2f-4306-b03e-5819ce825f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:58.971520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a70060-44b2-4f5b-a6a3-277ef1c4cd86 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.016131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9fa38-d0e4-4f93-8694-b50ce308f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.017098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9fa38-d0e4-4f93-8694-b50ce308f0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.017386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9fa38-d0e4-4f93-8694-b50ce308f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.017912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9fa38-d0e4-4f93-8694-b50ce308f0c3 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.019175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9fa38-d0e4-4f93-8694-b50ce308f0c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:59.019790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9fa38-d0e4-4f93-8694-b50ce308f0c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.021464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9fa38-d0e4-4f93-8694-b50ce308f0c3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.023377 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 348/697] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:59.024372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396b887-9e2f-4306-b03e-5819ce825f7a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.043473 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 350/698] 10.222.0.65 () {56 vars in 1133 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:59.051515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9fa38-d0e4-4f93-8694-b50ce308f0c3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.084693 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:59.132229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:59.163823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9fa38-d0e4-4f93-8694-b50ce308f0c3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.165201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396b887-9e2f-4306-b03e-5819ce825f7a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.199030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faba5f-69d6-488d-b737-9a1646c6b17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.200158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faba5f-69d6-488d-b737-9a1646c6b17f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.200695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faba5f-69d6-488d-b737-9a1646c6b17f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.201674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faba5f-69d6-488d-b737-9a1646c6b17f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.202996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fb8a9-6294-4532-806a-77f9b22126ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.203676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fb8a9-6294-4532-806a-77f9b22126ee None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.203972 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fb8a9-6294-4532-806a-77f9b22126ee None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.204253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fb8a9-6294-4532-806a-77f9b22126ee None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.205153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fb8a9-6294-4532-806a-77f9b22126ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:59.205475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fb8a9-6294-4532-806a-77f9b22126ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.207004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fb8a9-6294-4532-806a-77f9b22126ee None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.219943 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 349/699] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:59.220827 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fb8a9-6294-4532-806a-77f9b22126ee None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.274479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faba5f-69d6-488d-b737-9a1646c6b17f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.276341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faba5f-69d6-488d-b737-9a1646c6b17f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.307235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:59.333614 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fb8a9-6294-4532-806a-77f9b22126ee None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.366536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:59.373918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5468105-c2d9-4500-b78e-d73960914f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.375077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5468105-c2d9-4500-b78e-d73960914f3c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.375549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5468105-c2d9-4500-b78e-d73960914f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.376055 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5468105-c2d9-4500-b78e-d73960914f3c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.377030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5468105-c2d9-4500-b78e-d73960914f3c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.378715 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5468105-c2d9-4500-b78e-d73960914f3c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.387679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 351/700] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:59.387679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faba5f-69d6-488d-b737-9a1646c6b17f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c5468105-c2d9-4500-b78e-d73960914f3c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:59.402798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:59.404088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:49:59.405180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:59.407832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 350/701] 10.222.0.65 () {56 vars in 1129 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:59.408896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5468105-c2d9-4500-b78e-d73960914f3c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.502328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faba5f-69d6-488d-b737-9a1646c6b17f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.537407 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5468105-c2d9-4500-b78e-d73960914f3c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.544118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:59.547387 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a99194-c14c-4ba9-aed8-78ebc207d1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.548260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a99194-c14c-4ba9-aed8-78ebc207d1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.548691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a99194-c14c-4ba9-aed8-78ebc207d1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.549256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a99194-c14c-4ba9-aed8-78ebc207d1f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.550776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a99194-c14c-4ba9-aed8-78ebc207d1f7 None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=d3d8f8acd0484857b451877baf03e895) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:59.551263 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a99194-c14c-4ba9-aed8-78ebc207d1f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.553339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a99194-c14c-4ba9-aed8-78ebc207d1f7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.562763 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 352/702] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/role_assignments?user.id=d3d8f8acd0484857b451877baf03e895&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:59.564021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a99194-c14c-4ba9-aed8-78ebc207d1f7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.580981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5388f-565b-4d0a-ba0b-f0761655792b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.581942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5388f-565b-4d0a-ba0b-f0761655792b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.582328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5388f-565b-4d0a-ba0b-f0761655792b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.582771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5388f-565b-4d0a-ba0b-f0761655792b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.594703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5388f-565b-4d0a-ba0b-f0761655792b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.596521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5388f-565b-4d0a-ba0b-f0761655792b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7da5388f-565b-4d0a-ba0b-f0761655792b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:59.619777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:59.620736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:59.621634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:59.621634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:59.621634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:59.621634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:49:59.621634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 351/703] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:59.624409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5388f-565b-4d0a-ba0b-f0761655792b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.752490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:59.770778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a99194-c14c-4ba9-aed8-78ebc207d1f7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.814387 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd700da-e83d-4d7c-8326-c4a99c0622f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.815068 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd700da-e83d-4d7c-8326-c4a99c0622f0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.815515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd700da-e83d-4d7c-8326-c4a99c0622f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.815858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd700da-e83d-4d7c-8326-c4a99c0622f0 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.816571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd700da-e83d-4d7c-8326-c4a99c0622f0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:59.816881 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd700da-e83d-4d7c-8326-c4a99c0622f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.817672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5388f-565b-4d0a-ba0b-f0761655792b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.818261 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd700da-e83d-4d7c-8326-c4a99c0622f0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.823327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 353/704] 10.222.0.65 () {56 vars in 1124 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:59.828715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd700da-e83d-4d7c-8326-c4a99c0622f0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.856507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05ba2f-8bd1-4fa0-bdf9-abebba22379a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.858130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ba2f-8bd1-4fa0-bdf9-abebba22379a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.858419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ba2f-8bd1-4fa0-bdf9-abebba22379a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.858694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ba2f-8bd1-4fa0-bdf9-abebba22379a None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.859487 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05ba2f-8bd1-4fa0-bdf9-abebba22379a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:59.859949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05ba2f-8bd1-4fa0-bdf9-abebba22379a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.861207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05ba2f-8bd1-4fa0-bdf9-abebba22379a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.875514 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 352/705] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:59.876813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6182f-aae9-4d5c-baa1-0b999a14273e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.877102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6182f-aae9-4d5c-baa1-0b999a14273e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.877378 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6182f-aae9-4d5c-baa1-0b999a14273e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.878879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 353/706] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:59.885031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6182f-aae9-4d5c-baa1-0b999a14273e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.903699 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:59.954483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd700da-e83d-4d7c-8326-c4a99c0622f0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.998570 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e3764-5028-48fa-81a9-222be21e7725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.999495 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e3764-5028-48fa-81a9-222be21e7725 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.000051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e3764-5028-48fa-81a9-222be21e7725 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.000531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e3764-5028-48fa-81a9-222be21e7725 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.001472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e3764-5028-48fa-81a9-222be21e7725 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:00.001923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e3764-5028-48fa-81a9-222be21e7725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.003628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e3764-5028-48fa-81a9-222be21e7725 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.010665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 354/707] 10.222.0.65 () {56 vars in 1124 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:00.012288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2434253-379a-4e8d-b643-286b7cc244ee None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.012846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2434253-379a-4e8d-b643-286b7cc244ee None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.013887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2434253-379a-4e8d-b643-286b7cc244ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.018764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6182f-aae9-4d5c-baa1-0b999a14273e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.038611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b2434253-379a-4e8d-b643-286b7cc244ee None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:00.041632 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b2434253-379a-4e8d-b643-286b7cc244ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:00.042348 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:00.108547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c1e23-2212-48b6-a637-19b1e694550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.108547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c1e23-2212-48b6-a637-19b1e694550d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.108547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c1e23-2212-48b6-a637-19b1e694550d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.108547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c1e23-2212-48b6-a637-19b1e694550d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.120008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c1e23-2212-48b6-a637-19b1e694550d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.121985 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c1e23-2212-48b6-a637-19b1e694550d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-597c1e23-2212-48b6-a637-19b1e694550d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:00.152128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:00.153066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:00.154013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:00.154013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:00.154013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:00.154013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:00.154013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:00.154013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:00.154013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:00.154013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:00.161051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 354/708] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:00.162018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c1e23-2212-48b6-a637-19b1e694550d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.192432 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 355/709] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:00.219558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2434253-379a-4e8d-b643-286b7cc244ee None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.290851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:00.297909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c1e23-2212-48b6-a637-19b1e694550d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.333774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855463d-e530-436f-b954-c352e43d06a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.334440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855463d-e530-436f-b954-c352e43d06a7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.334722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855463d-e530-436f-b954-c352e43d06a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.334997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855463d-e530-436f-b954-c352e43d06a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.336680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855463d-e530-436f-b954-c352e43d06a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:00.337003 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855463d-e530-436f-b954-c352e43d06a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.338429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855463d-e530-436f-b954-c352e43d06a7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.341908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2434253-379a-4e8d-b643-286b7cc244ee None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.351141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 355/710] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:00.360654 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855463d-e530-436f-b954-c352e43d06a7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.390582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad250a1-fc95-472b-97ff-0dad0dcae672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.391692 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad250a1-fc95-472b-97ff-0dad0dcae672 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.391692 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad250a1-fc95-472b-97ff-0dad0dcae672 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.391972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad250a1-fc95-472b-97ff-0dad0dcae672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.412732 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad250a1-fc95-472b-97ff-0dad0dcae672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.415135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad250a1-fc95-472b-97ff-0dad0dcae672 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.416011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-bad250a1-fc95-472b-97ff-0dad0dcae672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:00.437895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:00.438860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:00.439788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:00.439788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:00.439788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:00.439788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:00.439788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 356/711] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:00.443678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2073dd-be8c-4184-b100-0c7950d4587a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.443678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2073dd-be8c-4184-b100-0c7950d4587a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.443678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2073dd-be8c-4184-b100-0c7950d4587a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.447098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 357/712] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:00.447933 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2073dd-be8c-4184-b100-0c7950d4587a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.507430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855463d-e530-436f-b954-c352e43d06a7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.556615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82415279-3793-4305-9bf6-1868dba711d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.557547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82415279-3793-4305-9bf6-1868dba711d3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.557836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82415279-3793-4305-9bf6-1868dba711d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.558118 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82415279-3793-4305-9bf6-1868dba711d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.559608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82415279-3793-4305-9bf6-1868dba711d3 None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=8458544c021049c587856343e8001429, user.id=b01d875111bd43698bcf99d30f7a7f9a) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:00.560072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82415279-3793-4305-9bf6-1868dba711d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.561716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82415279-3793-4305-9bf6-1868dba711d3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.562979 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2073dd-be8c-4184-b100-0c7950d4587a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.569062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 356/713] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/role_assignments?user.id=b01d875111bd43698bcf99d30f7a7f9a&scope.project.id=8458544c021049c587856343e8001429&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:00.570173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759c309-fe88-475c-adbb-ce41e37445b7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.570527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759c309-fe88-475c-adbb-ce41e37445b7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.571434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:00.572300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759c309-fe88-475c-adbb-ce41e37445b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.593973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d759c309-fe88-475c-adbb-ce41e37445b7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:00.595280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d759c309-fe88-475c-adbb-ce41e37445b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:00.658838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbad66-2f52-41f6-bdf0-c5f48da1c95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.659522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbad66-2f52-41f6-bdf0-c5f48da1c95c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.659820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbad66-2f52-41f6-bdf0-c5f48da1c95c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.660095 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbad66-2f52-41f6-bdf0-c5f48da1c95c None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.660826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbad66-2f52-41f6-bdf0-c5f48da1c95c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:00.661130 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbad66-2f52-41f6-bdf0-c5f48da1c95c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.662508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbad66-2f52-41f6-bdf0-c5f48da1c95c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.688030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 357/714] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:00.695115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 358/715] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:00.704772 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759c309-fe88-475c-adbb-ce41e37445b7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.711419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbad66-2f52-41f6-bdf0-c5f48da1c95c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.745213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:00.834021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbad66-2f52-41f6-bdf0-c5f48da1c95c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.842619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759c309-fe88-475c-adbb-ce41e37445b7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.878427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7b512-da0b-4b1f-8e84-a540108527cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.879275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7b512-da0b-4b1f-8e84-a540108527cf None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.879679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7b512-da0b-4b1f-8e84-a540108527cf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.880079 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7b512-da0b-4b1f-8e84-a540108527cf None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.881237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7b512-da0b-4b1f-8e84-a540108527cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.883194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7b512-da0b-4b1f-8e84-a540108527cf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.908676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 358/716] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:50:00 2022] POST /identity/v3/users => generated 254 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:00.937653 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed71cb-3c53-4d83-91a4-915aa38bfd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.938836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed71cb-3c53-4d83-91a4-915aa38bfd51 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.939276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed71cb-3c53-4d83-91a4-915aa38bfd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.939738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed71cb-3c53-4d83-91a4-915aa38bfd51 None admin] PATH_INFO: `/v3/services/network` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.940581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed71cb-3c53-4d83-91a4-915aa38bfd51 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.942207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed71cb-3c53-4d83-91a4-915aa38bfd51 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-01ed71cb-3c53-4d83-91a4-915aa38bfd51 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:00.963210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:00.964578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:00.965506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:00.973677 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 359/717] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/services/network => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:00.975402 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7b512-da0b-4b1f-8e84-a540108527cf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:01.084718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:01.108402 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7b512-da0b-4b1f-8e84-a540108527cf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:01.143625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d17ba5-cb20-4da0-941d-6d0dac592fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:01.144324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d17ba5-cb20-4da0-941d-6d0dac592fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.144612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d17ba5-cb20-4da0-941d-6d0dac592fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.144897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d17ba5-cb20-4da0-941d-6d0dac592fe9 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.145613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d17ba5-cb20-4da0-941d-6d0dac592fe9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:01.145922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d17ba5-cb20-4da0-941d-6d0dac592fe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:01.147302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d17ba5-cb20-4da0-941d-6d0dac592fe9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:01.152263 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 359/718] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:01.160710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed71cb-3c53-4d83-91a4-915aa38bfd51 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:01.193907 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:01.273768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed71cb-3c53-4d83-91a4-915aa38bfd51 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:01.309388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a19dd-849d-418f-bc5c-0977e40a3b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:01.310958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a19dd-849d-418f-bc5c-0977e40a3b22 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.311442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a19dd-849d-418f-bc5c-0977e40a3b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.311953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a19dd-849d-418f-bc5c-0977e40a3b22 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.312907 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a19dd-849d-418f-bc5c-0977e40a3b22 None admin] RBAC: Adding query filter params (type=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:01.313421 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a19dd-849d-418f-bc5c-0977e40a3b22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:01.314975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a19dd-849d-418f-bc5c-0977e40a3b22 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:01.323759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 360/719] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:02.156835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c402ca-ffc0-4c38-869b-c432256f1923 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.157368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c402ca-ffc0-4c38-869b-c432256f1923 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.157368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c402ca-ffc0-4c38-869b-c432256f1923 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.159030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 360/720] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:02.164110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d713b-4a14-44ac-b9b9-89c8aa4ce4f6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.164525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d713b-4a14-44ac-b9b9-89c8aa4ce4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.164703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d713b-4a14-44ac-b9b9-89c8aa4ce4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.192613 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-575d713b-4a14-44ac-b9b9-89c8aa4ce4f6 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:02.193920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-575d713b-4a14-44ac-b9b9-89c8aa4ce4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:02.279794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 361/721] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:02.321796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c402ca-ffc0-4c38-869b-c432256f1923 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.437136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c402ca-ffc0-4c38-869b-c432256f1923 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.471885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844ade8-cc99-42d6-b333-6f8fd889ae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.472689 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844ade8-cc99-42d6-b333-6f8fd889ae6c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.473038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844ade8-cc99-42d6-b333-6f8fd889ae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.473320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844ade8-cc99-42d6-b333-6f8fd889ae6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.485116 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844ade8-cc99-42d6-b333-6f8fd889ae6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f844ade8-cc99-42d6-b333-6f8fd889ae6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:02.493093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:02.494071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:02.494928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 361/722] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:02.500660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d713b-4a14-44ac-b9b9-89c8aa4ce4f6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.518768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362b8f9-f30c-44f6-91fb-2db26033c720 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.519336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362b8f9-f30c-44f6-91fb-2db26033c720 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.519830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362b8f9-f30c-44f6-91fb-2db26033c720 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.521628 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 362/723] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:02.529174 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf12c-2701-452e-b93e-b78c5262c063 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.529758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf12c-2701-452e-b93e-b78c5262c063 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.530278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf12c-2701-452e-b93e-b78c5262c063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.534286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:02.535674 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:02.549041 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-25dbf12c-2701-452e-b93e-b78c5262c063 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:02.550516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-25dbf12c-2701-452e-b93e-b78c5262c063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:02.625061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 363/724] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:50:02.659792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbf12c-2701-452e-b93e-b78c5262c063 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.686224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d713b-4a14-44ac-b9b9-89c8aa4ce4f6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.779401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbf12c-2701-452e-b93e-b78c5262c063 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.812881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:02.816466 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8ad81-6ffd-4af4-9626-bee60ec20df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.817329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1781cc-8504-4782-a89d-8ac789b438c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.817491 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8ad81-6ffd-4af4-9626-bee60ec20df1 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.817912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8ad81-6ffd-4af4-9626-bee60ec20df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.818288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1781cc-8504-4782-a89d-8ac789b438c3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.818500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8ad81-6ffd-4af4-9626-bee60ec20df1 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.818753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1781cc-8504-4782-a89d-8ac789b438c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.819124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1781cc-8504-4782-a89d-8ac789b438c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.819711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8ad81-6ffd-4af4-9626-bee60ec20df1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:02.820098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1781cc-8504-4782-a89d-8ac789b438c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:02.820502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1781cc-8504-4782-a89d-8ac789b438c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:02.821578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8ad81-6ffd-4af4-9626-bee60ec20df1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:02.822059 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1781cc-8504-4782-a89d-8ac789b438c3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:02.840890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 364/725] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:02 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:50:02.843866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 362/726] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:02.855425 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8ad81-6ffd-4af4-9626-bee60ec20df1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.988680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8ad81-6ffd-4af4-9626-bee60ec20df1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.011790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d4b24-55af-45cc-8dc3-37d037a0cab4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.012321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d4b24-55af-45cc-8dc3-37d037a0cab4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.012815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d4b24-55af-45cc-8dc3-37d037a0cab4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.014794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 363/727] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:03.025782 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daef385-56c2-4d98-bf62-28bf9db38d53 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.026475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daef385-56c2-4d98-bf62-28bf9db38d53 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.026957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daef385-56c2-4d98-bf62-28bf9db38d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.046736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e6fc9-ee62-4da0-800b-a221d81ae6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.047913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e6fc9-ee62-4da0-800b-a221d81ae6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.048543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e6fc9-ee62-4da0-800b-a221d81ae6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.049043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e6fc9-ee62-4da0-800b-a221d81ae6d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.057770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3daef385-56c2-4d98-bf62-28bf9db38d53 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:03.058978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3daef385-56c2-4d98-bf62-28bf9db38d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:03.060770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e6fc9-ee62-4da0-800b-a221d81ae6d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.062446 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e6fc9-ee62-4da0-800b-a221d81ae6d3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ad2e6fc9-ee62-4da0-800b-a221d81ae6d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:03.080933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:03.081866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:03.082789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:03.082789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:03.082789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:03.082789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:03.085032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 365/728] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:03.090261 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e6fc9-ee62-4da0-800b-a221d81ae6d3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.144998 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 364/729] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:03.150026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:03.186131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daef385-56c2-4d98-bf62-28bf9db38d53 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.213008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:03.225160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e6fc9-ee62-4da0-800b-a221d81ae6d3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.290760 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53794ff-780a-449f-a44f-e07bc30a5a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.291439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53794ff-780a-449f-a44f-e07bc30a5a15 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.291747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53794ff-780a-449f-a44f-e07bc30a5a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.292027 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53794ff-780a-449f-a44f-e07bc30a5a15 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.292761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53794ff-780a-449f-a44f-e07bc30a5a15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:03.293068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53794ff-780a-449f-a44f-e07bc30a5a15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.300236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53794ff-780a-449f-a44f-e07bc30a5a15 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.322714 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 366/730] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:03.329137 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53794ff-780a-449f-a44f-e07bc30a5a15 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.368810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daef385-56c2-4d98-bf62-28bf9db38d53 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.412711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0a3fb-9b95-470c-96c9-dfb9c6a433e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.413584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0a3fb-9b95-470c-96c9-dfb9c6a433e1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.413869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0a3fb-9b95-470c-96c9-dfb9c6a433e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.414201 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0a3fb-9b95-470c-96c9-dfb9c6a433e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.442240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0a3fb-9b95-470c-96c9-dfb9c6a433e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-46d0a3fb-9b95-470c-96c9-dfb9c6a433e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:03.449988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:03.451137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:03.452041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:03.452041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:03.452041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:03.452041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:03.452041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:03.452041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:03.452041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:03.452041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:03.452041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:03.452041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:03.452041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:03.452041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:03.454412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 365/731] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:03.460587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0a3fb-9b95-470c-96c9-dfb9c6a433e1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.498660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53794ff-780a-449f-a44f-e07bc30a5a15 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.520423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:03.536440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f68c2-40cf-48b7-bfe7-a8214ddcd6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.537245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f68c2-40cf-48b7-bfe7-a8214ddcd6ca None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.537533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f68c2-40cf-48b7-bfe7-a8214ddcd6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.537812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f68c2-40cf-48b7-bfe7-a8214ddcd6ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.545800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f68c2-40cf-48b7-bfe7-a8214ddcd6ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.547269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f68c2-40cf-48b7-bfe7-a8214ddcd6ca None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7c0f68c2-40cf-48b7-bfe7-a8214ddcd6ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:03.562655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:03.563634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:03.564610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:03.564610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:03.564610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:03.564610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:03.564610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:03.564610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:03.564610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:03.564610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:03.564610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:03.564610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:03.564610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 367/732] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:03.574611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f68c2-40cf-48b7-bfe7-a8214ddcd6ca None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.613037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:03.659477 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0a3fb-9b95-470c-96c9-dfb9c6a433e1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.722250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f68c2-40cf-48b7-bfe7-a8214ddcd6ca None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.725932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67453f-3f7d-47ad-8e69-951395f44c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.726882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67453f-3f7d-47ad-8e69-951395f44c12 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.727376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67453f-3f7d-47ad-8e69-951395f44c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.727905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67453f-3f7d-47ad-8e69-951395f44c12 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.728855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67453f-3f7d-47ad-8e69-951395f44c12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:03.729457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67453f-3f7d-47ad-8e69-951395f44c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.731041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67453f-3f7d-47ad-8e69-951395f44c12 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.741184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 366/733] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:03.742651 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc014be1-9a49-4583-b5e6-2bfb605895d0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.743143 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc014be1-9a49-4583-b5e6-2bfb605895d0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.743639 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc014be1-9a49-4583-b5e6-2bfb605895d0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.745672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 367/734] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:03.746567 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc014be1-9a49-4583-b5e6-2bfb605895d0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.760089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491155f-b6fb-46da-a378-7d991b1c00d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.761096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491155f-b6fb-46da-a378-7d991b1c00d0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.761584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491155f-b6fb-46da-a378-7d991b1c00d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.762086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491155f-b6fb-46da-a378-7d991b1c00d0 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.763180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491155f-b6fb-46da-a378-7d991b1c00d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:03.764222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491155f-b6fb-46da-a378-7d991b1c00d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.765858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491155f-b6fb-46da-a378-7d991b1c00d0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.784905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 368/735] 10.222.0.65 () {56 vars in 1135 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:03.786530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f5524-b1c6-467c-bb85-b96e4864abb5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.791065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f5524-b1c6-467c-bb85-b96e4864abb5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.791354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f5524-b1c6-467c-bb85-b96e4864abb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.808149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-388f5524-b1c6-467c-bb85-b96e4864abb5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:03.809376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-388f5524-b1c6-467c-bb85-b96e4864abb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:03.823567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:03.875631 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc014be1-9a49-4583-b5e6-2bfb605895d0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.894329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 369/736] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:03.895346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f5524-b1c6-467c-bb85-b96e4864abb5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.918847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c019b-03e8-4cb3-a34f-5cf3df320c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.920497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c019b-03e8-4cb3-a34f-5cf3df320c34 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.921409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c019b-03e8-4cb3-a34f-5cf3df320c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.921838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c019b-03e8-4cb3-a34f-5cf3df320c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.933066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:03.940081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c019b-03e8-4cb3-a34f-5cf3df320c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.941706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c019b-03e8-4cb3-a34f-5cf3df320c34 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-bb0c019b-03e8-4cb3-a34f-5cf3df320c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:03.966558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:03.967680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:03.968685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:03.968685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:03.968685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:03.968685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:03.970077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 368/737] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:03.972289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c019b-03e8-4cb3-a34f-5cf3df320c34 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.020151 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f5524-b1c6-467c-bb85-b96e4864abb5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.061743 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25f8fc-9cdb-4f49-bb47-a9c13fa1cfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.062401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25f8fc-9cdb-4f49-bb47-a9c13fa1cfda None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.062688 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25f8fc-9cdb-4f49-bb47-a9c13fa1cfda None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.062967 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25f8fc-9cdb-4f49-bb47-a9c13fa1cfda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.099199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c019b-03e8-4cb3-a34f-5cf3df320c34 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.138088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af83c2-1c45-4501-b9cd-20edb89835f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.138945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af83c2-1c45-4501-b9cd-20edb89835f2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.139227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af83c2-1c45-4501-b9cd-20edb89835f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.139500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af83c2-1c45-4501-b9cd-20edb89835f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.151594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af83c2-1c45-4501-b9cd-20edb89835f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c4af83c2-1c45-4501-b9cd-20edb89835f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:04.159566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:04.160750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:04.161610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 369/738] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:04.162323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af83c2-1c45-4501-b9cd-20edb89835f2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.196867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.221100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.251729 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25f8fc-9cdb-4f49-bb47-a9c13fa1cfda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.253203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25f8fc-9cdb-4f49-bb47-a9c13fa1cfda None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.276444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af83c2-1c45-4501-b9cd-20edb89835f2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.317049 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfc856-3c9b-4b20-b1cd-0e2cda48c4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.318024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfc856-3c9b-4b20-b1cd-0e2cda48c4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.318453 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfc856-3c9b-4b20-b1cd-0e2cda48c4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.318848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfc856-3c9b-4b20-b1cd-0e2cda48c4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.319576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfc856-3c9b-4b20-b1cd-0e2cda48c4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:04.319917 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfc856-3c9b-4b20-b1cd-0e2cda48c4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.321477 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfc856-3c9b-4b20-b1cd-0e2cda48c4d3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.340898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 370/739] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:04.340898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13460bc2-fdaa-4aea-88f4-deaebb42c04f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.340898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13460bc2-fdaa-4aea-88f4-deaebb42c04f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.340898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13460bc2-fdaa-4aea-88f4-deaebb42c04f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.342281 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 371/740] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:04.342832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13460bc2-fdaa-4aea-88f4-deaebb42c04f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.363542 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 370/741] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:50:04.364732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25f8fc-9cdb-4f49-bb47-a9c13fa1cfda None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.415604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.456134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13460bc2-fdaa-4aea-88f4-deaebb42c04f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.469483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.471912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25f8fc-9cdb-4f49-bb47-a9c13fa1cfda None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.496893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507034df-19a1-419d-84a5-d292a1826c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.497706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507034df-19a1-419d-84a5-d292a1826c52 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.497993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507034df-19a1-419d-84a5-d292a1826c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.498377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507034df-19a1-419d-84a5-d292a1826c52 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.499130 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507034df-19a1-419d-84a5-d292a1826c52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:04.499443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507034df-19a1-419d-84a5-d292a1826c52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.501202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507034df-19a1-419d-84a5-d292a1826c52 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.505899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e63fa-2268-4ded-a173-a744aebcff79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.506680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e63fa-2268-4ded-a173-a744aebcff79 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.506961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e63fa-2268-4ded-a173-a744aebcff79 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.507229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e63fa-2268-4ded-a173-a744aebcff79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.509932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 372/742] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:04.511129 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e7886-4f24-4c47-961e-022df78bc8ae None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.511421 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e7886-4f24-4c47-961e-022df78bc8ae None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.511875 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e7886-4f24-4c47-961e-022df78bc8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.518503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e63fa-2268-4ded-a173-a744aebcff79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.519961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e63fa-2268-4ded-a173-a744aebcff79 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.531097 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d94e7886-4f24-4c47-961e-022df78bc8ae None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:04.532345 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d94e7886-4f24-4c47-961e-022df78bc8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ce7e63fa-2268-4ded-a173-a744aebcff79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:04.534436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:04.535353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:04.536357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:04.536357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:04.536357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:04.536357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:04.536357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:04.536357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:04.536357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:04.536357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:04.536357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:04.536357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:04.536357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 371/743] 10.222.0.65 () {56 vars in 1093 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:04.536357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e63fa-2268-4ded-a173-a744aebcff79 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.613256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 373/744] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:04.613256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e7886-4f24-4c47-961e-022df78bc8ae None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.642753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e63fa-2268-4ded-a173-a744aebcff79 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.651852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.679304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ebcb0-b191-4049-863c-fe7098f45b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.680211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ebcb0-b191-4049-863c-fe7098f45b21 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.680493 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ebcb0-b191-4049-863c-fe7098f45b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.680767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ebcb0-b191-4049-863c-fe7098f45b21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.692025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ebcb0-b191-4049-863c-fe7098f45b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.693595 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ebcb0-b191-4049-863c-fe7098f45b21 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-232ebcb0-b191-4049-863c-fe7098f45b21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:04.712042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:04.713177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:04.714320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:04.714320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:04.714320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:04.714320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:04.714320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 372/745] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:04.714320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ebcb0-b191-4049-863c-fe7098f45b21 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.725257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e7886-4f24-4c47-961e-022df78bc8ae None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.726812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.760461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc4b52-5e69-4b17-826d-6f6dac66063a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.761258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc4b52-5e69-4b17-826d-6f6dac66063a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.761547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc4b52-5e69-4b17-826d-6f6dac66063a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.761829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc4b52-5e69-4b17-826d-6f6dac66063a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.773055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc4b52-5e69-4b17-826d-6f6dac66063a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.774949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc4b52-5e69-4b17-826d-6f6dac66063a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-aebc4b52-5e69-4b17-826d-6f6dac66063a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:04.794177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:04.795137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:04.800966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:04.800966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:04.800966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:04.800966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:04.800966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 374/746] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:04.800966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc4b52-5e69-4b17-826d-6f6dac66063a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.828094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ebcb0-b191-4049-863c-fe7098f45b21 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.865211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f3d14-222a-466a-990f-0c0094dc66d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.865887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f3d14-222a-466a-990f-0c0094dc66d6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.866171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f3d14-222a-466a-990f-0c0094dc66d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.866455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f3d14-222a-466a-990f-0c0094dc66d6 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.867337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f3d14-222a-466a-990f-0c0094dc66d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:04.867664 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f3d14-222a-466a-990f-0c0094dc66d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.869365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f3d14-222a-466a-990f-0c0094dc66d6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.889541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 373/747] 10.222.0.65 () {56 vars in 1133 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:04.890372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f3d14-222a-466a-990f-0c0094dc66d6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.909859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc4b52-5e69-4b17-826d-6f6dac66063a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.911211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.947855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d0da9-e847-4b53-81ca-3a522e3ce9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.948520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d0da9-e847-4b53-81ca-3a522e3ce9af None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.948821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d0da9-e847-4b53-81ca-3a522e3ce9af None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.949074 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d0da9-e847-4b53-81ca-3a522e3ce9af None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.949791 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d0da9-e847-4b53-81ca-3a522e3ce9af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.951715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d0da9-e847-4b53-81ca-3a522e3ce9af None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.975828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 375/748] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:04 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:04.975828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d0da9-e847-4b53-81ca-3a522e3ce9af None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.995398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.011553 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f3d14-222a-466a-990f-0c0094dc66d6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.065230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2dfa2-89fe-4818-8e38-c6beebf951a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.066716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2dfa2-89fe-4818-8e38-c6beebf951a4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.067189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2dfa2-89fe-4818-8e38-c6beebf951a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.067735 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2dfa2-89fe-4818-8e38-c6beebf951a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.069826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2dfa2-89fe-4818-8e38-c6beebf951a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:05.070361 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2dfa2-89fe-4818-8e38-c6beebf951a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.072025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2dfa2-89fe-4818-8e38-c6beebf951a4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.093226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 374/749] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:05.094506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2dfa2-89fe-4818-8e38-c6beebf951a4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.119688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d0da9-e847-4b53-81ca-3a522e3ce9af None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.169051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ddf01-4d82-4a29-bbce-145b9e1e5251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.170226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ddf01-4d82-4a29-bbce-145b9e1e5251 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.170515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ddf01-4d82-4a29-bbce-145b9e1e5251 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.170949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ddf01-4d82-4a29-bbce-145b9e1e5251 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.173081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ddf01-4d82-4a29-bbce-145b9e1e5251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:05.173081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ddf01-4d82-4a29-bbce-145b9e1e5251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.173544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ddf01-4d82-4a29-bbce-145b9e1e5251 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.187836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 376/750] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:05.188564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ddf01-4d82-4a29-bbce-145b9e1e5251 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.201902 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.227952 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2dfa2-89fe-4818-8e38-c6beebf951a4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.267070 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b8046-7807-4a71-b78f-c0fa352c8e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.267926 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b8046-7807-4a71-b78f-c0fa352c8e03 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.268209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b8046-7807-4a71-b78f-c0fa352c8e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.268483 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b8046-7807-4a71-b78f-c0fa352c8e03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.274034 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.279956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b8046-7807-4a71-b78f-c0fa352c8e03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.281402 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b8046-7807-4a71-b78f-c0fa352c8e03 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f20b8046-7807-4a71-b78f-c0fa352c8e03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:05.298716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:05.299856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:05.300898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:05.300898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:05.300898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:05.300898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:05.300898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 375/751] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:05.301244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b8046-7807-4a71-b78f-c0fa352c8e03 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.307189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ddf01-4d82-4a29-bbce-145b9e1e5251 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.345318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc24802-f69c-4ad7-bf65-541c69bf1c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.346127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc24802-f69c-4ad7-bf65-541c69bf1c27 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.346410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc24802-f69c-4ad7-bf65-541c69bf1c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.346684 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc24802-f69c-4ad7-bf65-541c69bf1c27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.354379 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc24802-f69c-4ad7-bf65-541c69bf1c27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.355937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc24802-f69c-4ad7-bf65-541c69bf1c27 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-acc24802-f69c-4ad7-bf65-541c69bf1c27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:05.370574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:05.371503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:05.373433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:05.373433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:05.373433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:05.373433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:05.373433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:05.373433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:05.373433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:05.373433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:05.373433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 377/752] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:05.373913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806467d-5db7-4c7d-b841-e6dc1bd78b72 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.373913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806467d-5db7-4c7d-b841-e6dc1bd78b72 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.374435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806467d-5db7-4c7d-b841-e6dc1bd78b72 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.375586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 378/753] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:05.376226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806467d-5db7-4c7d-b841-e6dc1bd78b72 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.420866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b8046-7807-4a71-b78f-c0fa352c8e03 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.448248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.460739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfda1d-8c6a-4018-9415-5186d66e734a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.462540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfda1d-8c6a-4018-9415-5186d66e734a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.462889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfda1d-8c6a-4018-9415-5186d66e734a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.463175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfda1d-8c6a-4018-9415-5186d66e734a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.472164 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfda1d-8c6a-4018-9415-5186d66e734a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.473603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfda1d-8c6a-4018-9415-5186d66e734a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f2cfda1d-8c6a-4018-9415-5186d66e734a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:05.489233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:05.490207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:05.491146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:05.491146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:05.491146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:05.491146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:05.491146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:05.491146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:05.491146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:05.491146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:05.491146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:05.491146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:05.491146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 376/754] 10.222.0.65 () {56 vars in 1117 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:05.491774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfda1d-8c6a-4018-9415-5186d66e734a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.505632 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806467d-5db7-4c7d-b841-e6dc1bd78b72 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.543093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c9039-0571-4e72-bef3-5bbcbd9b38b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.543862 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c9039-0571-4e72-bef3-5bbcbd9b38b8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.544149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c9039-0571-4e72-bef3-5bbcbd9b38b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.545118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c9039-0571-4e72-bef3-5bbcbd9b38b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.546098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c9039-0571-4e72-bef3-5bbcbd9b38b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:05.546411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c9039-0571-4e72-bef3-5bbcbd9b38b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.547943 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c9039-0571-4e72-bef3-5bbcbd9b38b8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.549656 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.567512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 379/755] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:05.567512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b8cc3-6bcc-4235-a23b-1aa36a785ab2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.567512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b8cc3-6bcc-4235-a23b-1aa36a785ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.567512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b8cc3-6bcc-4235-a23b-1aa36a785ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.583922 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1e1b8cc3-6bcc-4235-a23b-1aa36a785ab2 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:05.585246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1e1b8cc3-6bcc-4235-a23b-1aa36a785ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:05.602102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfda1d-8c6a-4018-9415-5186d66e734a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.637066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b55a2c-4297-42fc-9743-7c347e752e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.637737 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b55a2c-4297-42fc-9743-7c347e752e2a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.638014 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b55a2c-4297-42fc-9743-7c347e752e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.638314 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b55a2c-4297-42fc-9743-7c347e752e2a None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.639105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b55a2c-4297-42fc-9743-7c347e752e2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:05.639411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b55a2c-4297-42fc-9743-7c347e752e2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.640914 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b55a2c-4297-42fc-9743-7c347e752e2a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.653403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 377/756] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:05.654238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b55a2c-4297-42fc-9743-7c347e752e2a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.662476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 380/757] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:05.663595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b8cc3-6bcc-4235-a23b-1aa36a785ab2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.705944 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.767990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b55a2c-4297-42fc-9743-7c347e752e2a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.776299 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b8cc3-6bcc-4235-a23b-1aa36a785ab2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.802874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.804291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6fbe0-5177-4a8e-8804-64bafbd6cdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.804988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6fbe0-5177-4a8e-8804-64bafbd6cdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.805279 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6fbe0-5177-4a8e-8804-64bafbd6cdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.805560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6fbe0-5177-4a8e-8804-64bafbd6cdf6 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.806406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6fbe0-5177-4a8e-8804-64bafbd6cdf6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:05.806718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6fbe0-5177-4a8e-8804-64bafbd6cdf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.808172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6fbe0-5177-4a8e-8804-64bafbd6cdf6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.815349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49217631-c1e1-4533-9857-a6a444b075e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.816313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49217631-c1e1-4533-9857-a6a444b075e1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.816600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49217631-c1e1-4533-9857-a6a444b075e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.816957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49217631-c1e1-4533-9857-a6a444b075e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.823965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49217631-c1e1-4533-9857-a6a444b075e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.825418 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49217631-c1e1-4533-9857-a6a444b075e1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.828041 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 378/758] 10.222.0.65 () {56 vars in 1149 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:05.829020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6fbe0-5177-4a8e-8804-64bafbd6cdf6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-49217631-c1e1-4533-9857-a6a444b075e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:05.844973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:05.845929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:05.846825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:05.846825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:05.846825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:05.846825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:05.846825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:05.846825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:05.846825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:05.846825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:05.846825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 381/759] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:05.847515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49217631-c1e1-4533-9857-a6a444b075e1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.941845 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6fbe0-5177-4a8e-8804-64bafbd6cdf6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.945227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.964382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49217631-c1e1-4533-9857-a6a444b075e1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.981153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078c4ec-4207-4a85-a68f-148419360551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.982160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078c4ec-4207-4a85-a68f-148419360551 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.982650 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078c4ec-4207-4a85-a68f-148419360551 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.983363 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078c4ec-4207-4a85-a68f-148419360551 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.984558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078c4ec-4207-4a85-a68f-148419360551 None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=8458544c021049c587856343e8001429, user.id=d3d8f8acd0484857b451877baf03e895) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:05.984949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078c4ec-4207-4a85-a68f-148419360551 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.986379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078c4ec-4207-4a85-a68f-148419360551 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.999691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 379/760] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/role_assignments?user.id=d3d8f8acd0484857b451877baf03e895&scope.project.id=8458544c021049c587856343e8001429&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:05.999691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078c4ec-4207-4a85-a68f-148419360551 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.005805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664dfa6-9477-4467-8620-cef8b6a77ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.006763 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664dfa6-9477-4467-8620-cef8b6a77ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.007049 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664dfa6-9477-4467-8620-cef8b6a77ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.007327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664dfa6-9477-4467-8620-cef8b6a77ef3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.008098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664dfa6-9477-4467-8620-cef8b6a77ef3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.009522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664dfa6-9477-4467-8620-cef8b6a77ef3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e664dfa6-9477-4467-8620-cef8b6a77ef3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:06.023295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:06.024603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:06.025666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:06.029694 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 382/761] 10.222.0.65 () {56 vars in 1129 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:06.030650 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664dfa6-9477-4467-8620-cef8b6a77ef3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.102450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.136252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078c4ec-4207-4a85-a68f-148419360551 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.170677 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664dfa6-9477-4467-8620-cef8b6a77ef3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.183520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2aafa-b046-4906-bdbd-2ba4dc0ba542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.184209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2aafa-b046-4906-bdbd-2ba4dc0ba542 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.184593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2aafa-b046-4906-bdbd-2ba4dc0ba542 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.184765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2aafa-b046-4906-bdbd-2ba4dc0ba542 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.214874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7124065-b74f-4978-8743-1fbd0a0a7c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.215753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7124065-b74f-4978-8743-1fbd0a0a7c66 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.217072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7124065-b74f-4978-8743-1fbd0a0a7c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.217366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7124065-b74f-4978-8743-1fbd0a0a7c66 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.218449 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7124065-b74f-4978-8743-1fbd0a0a7c66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:06.218770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7124065-b74f-4978-8743-1fbd0a0a7c66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.220400 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7124065-b74f-4978-8743-1fbd0a0a7c66 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.228636 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.235095 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 383/762] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:06.236344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7124065-b74f-4978-8743-1fbd0a0a7c66 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.268222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2aafa-b046-4906-bdbd-2ba4dc0ba542 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.270444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2aafa-b046-4906-bdbd-2ba4dc0ba542 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.342513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7124065-b74f-4978-8743-1fbd0a0a7c66 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.352246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.378940 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 380/763] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:06.380118 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2aafa-b046-4906-bdbd-2ba4dc0ba542 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.383603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb75df-4a04-4383-9d05-3e8f021b66ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.384328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb75df-4a04-4383-9d05-3e8f021b66ee None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.385450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb75df-4a04-4383-9d05-3e8f021b66ee None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.385738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb75df-4a04-4383-9d05-3e8f021b66ee None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.386461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb75df-4a04-4383-9d05-3e8f021b66ee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:06.386772 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb75df-4a04-4383-9d05-3e8f021b66ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.389012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb75df-4a04-4383-9d05-3e8f021b66ee None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.399346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 384/764] 10.222.0.65 () {56 vars in 1124 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:06.400225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb75df-4a04-4383-9d05-3e8f021b66ee None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.458184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.496995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2aafa-b046-4906-bdbd-2ba4dc0ba542 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.525266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb75df-4a04-4383-9d05-3e8f021b66ee None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.533849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a049c3e-ca8a-410e-b35f-02fbb1b8921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.535061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a049c3e-ca8a-410e-b35f-02fbb1b8921f None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.535546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a049c3e-ca8a-410e-b35f-02fbb1b8921f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.536242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a049c3e-ca8a-410e-b35f-02fbb1b8921f None admin] PATH_INFO: `/v3/projects/8458544c021049c587856343e8001429/users/b01d875111bd43698bcf99d30f7a7f9a/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.538544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a049c3e-ca8a-410e-b35f-02fbb1b8921f None admin] RBAC: Authorizing `identity:create_grant(project_id=8458544c021049c587856343e8001429, user_id=b01d875111bd43698bcf99d30f7a7f9a, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.540309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a049c3e-ca8a-410e-b35f-02fbb1b8921f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.556411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 381/765] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:06 2022] PUT /identity/v3/projects/8458544c021049c587856343e8001429/users/b01d875111bd43698bcf99d30f7a7f9a/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:06.557918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a049c3e-ca8a-410e-b35f-02fbb1b8921f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.563716 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fe8a1-d75b-438b-ac77-2a069a227cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.564543 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fe8a1-d75b-438b-ac77-2a069a227cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.564826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fe8a1-d75b-438b-ac77-2a069a227cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.565105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fe8a1-d75b-438b-ac77-2a069a227cb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.576809 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fe8a1-d75b-438b-ac77-2a069a227cb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.578267 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fe8a1-d75b-438b-ac77-2a069a227cb0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-757fe8a1-d75b-438b-ac77-2a069a227cb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:06.597690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:06.598745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:06.599690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:06.599690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:06.599690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:06.599690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:06.603782 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 385/766] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:06.605635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fe8a1-d75b-438b-ac77-2a069a227cb0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.634528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.718345 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fe8a1-d75b-438b-ac77-2a069a227cb0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.733193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a049c3e-ca8a-410e-b35f-02fbb1b8921f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.766106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4aecd-7173-4d16-adf2-7b0a6b88edb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.766330 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4aecd-7173-4d16-adf2-7b0a6b88edb4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.766583 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4aecd-7173-4d16-adf2-7b0a6b88edb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.766963 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4aecd-7173-4d16-adf2-7b0a6b88edb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.767585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4aecd-7173-4d16-adf2-7b0a6b88edb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:06.767923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4aecd-7173-4d16-adf2-7b0a6b88edb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.769296 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4aecd-7173-4d16-adf2-7b0a6b88edb4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.775224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11363e4b-da94-4243-bd13-d7952d1cdfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.776217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11363e4b-da94-4243-bd13-d7952d1cdfde None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.776812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11363e4b-da94-4243-bd13-d7952d1cdfde None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.777345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11363e4b-da94-4243-bd13-d7952d1cdfde None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.778294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11363e4b-da94-4243-bd13-d7952d1cdfde None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:06.779430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11363e4b-da94-4243-bd13-d7952d1cdfde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.781174 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11363e4b-da94-4243-bd13-d7952d1cdfde None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.787894 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 386/767] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:06.788538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 382/768] 10.222.0.65 () {56 vars in 1124 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:06.791301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4aecd-7173-4d16-adf2-7b0a6b88edb4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.791443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.796122 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11363e4b-da94-4243-bd13-d7952d1cdfde None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.880722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.905260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4aecd-7173-4d16-adf2-7b0a6b88edb4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.911238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11363e4b-da94-4243-bd13-d7952d1cdfde None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.946911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd5c66-266a-4a1e-8b02-3eaee07d56ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.947993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd5c66-266a-4a1e-8b02-3eaee07d56ca None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.948481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd5c66-266a-4a1e-8b02-3eaee07d56ca None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.948957 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd5c66-266a-4a1e-8b02-3eaee07d56ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.949962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd5c66-266a-4a1e-8b02-3eaee07d56ca None admin] RBAC: Adding query filter params (interface=public, service_id=a84625a705b945b4b55758820655833c, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:06.950492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd5c66-266a-4a1e-8b02-3eaee07d56ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.952762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd5c66-266a-4a1e-8b02-3eaee07d56ca None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.961269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 383/769] 10.222.0.65 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/endpoints?service_id=a84625a705b945b4b55758820655833c&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:06.967018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8809c-d351-4dc6-8352-c4b53e7898db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.967520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd5c66-266a-4a1e-8b02-3eaee07d56ca None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.967854 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8809c-d351-4dc6-8352-c4b53e7898db None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.968138 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8809c-d351-4dc6-8352-c4b53e7898db None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.968411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8809c-d351-4dc6-8352-c4b53e7898db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.975633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8809c-d351-4dc6-8352-c4b53e7898db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.977099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8809c-d351-4dc6-8352-c4b53e7898db None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6ba8809c-d351-4dc6-8352-c4b53e7898db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:07.006030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:07.006965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:07.008355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:07.008355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:07.008355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:07.008355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:07.008355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:07.008355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:07.008355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:07.008355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:07.008786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 387/770] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:07.016670 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8809c-d351-4dc6-8352-c4b53e7898db None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.032022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:07.079491 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd5c66-266a-4a1e-8b02-3eaee07d56ca None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.116232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218693a-bb08-44be-b843-ee1522259633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.117341 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218693a-bb08-44be-b843-ee1522259633 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.117825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218693a-bb08-44be-b843-ee1522259633 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.118315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218693a-bb08-44be-b843-ee1522259633 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.119212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218693a-bb08-44be-b843-ee1522259633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:07.120831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218693a-bb08-44be-b843-ee1522259633 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.126868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 384/771] 10.222.0.65 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/services => generated 772 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:07.241033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8809c-d351-4dc6-8352-c4b53e7898db None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.288861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a50df-22e9-4597-a984-b25e8cf890d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.289856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a50df-22e9-4597-a984-b25e8cf890d8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.290397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a50df-22e9-4597-a984-b25e8cf890d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.290889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a50df-22e9-4597-a984-b25e8cf890d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.292203 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a50df-22e9-4597-a984-b25e8cf890d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:07.292798 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a50df-22e9-4597-a984-b25e8cf890d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:07.294467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a50df-22e9-4597-a984-b25e8cf890d8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.308262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 388/772] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:07.314662 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218693a-bb08-44be-b843-ee1522259633 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.406108 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e45da5-2a03-4b00-8106-56b0c28b124e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.407137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e45da5-2a03-4b00-8106-56b0c28b124e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.407638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e45da5-2a03-4b00-8106-56b0c28b124e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.409248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 389/773] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:07.414328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2fa95-8451-42d4-bf7d-add8b344ad66 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.414876 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2fa95-8451-42d4-bf7d-add8b344ad66 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.415356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2fa95-8451-42d4-bf7d-add8b344ad66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.422588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:07.431743 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218693a-bb08-44be-b843-ee1522259633 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.436455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9eb2fa95-8451-42d4-bf7d-add8b344ad66 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:07.437880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9eb2fa95-8451-42d4-bf7d-add8b344ad66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:07.464178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:07.489576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8140c-9dcf-4a40-9720-0cd173209cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.490241 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8140c-9dcf-4a40-9720-0cd173209cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.490521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8140c-9dcf-4a40-9720-0cd173209cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.490794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8140c-9dcf-4a40-9720-0cd173209cd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.491631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8140c-9dcf-4a40-9720-0cd173209cd7 None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=96655b920bda45cb872624aa4183cb58) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:07.491978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8140c-9dcf-4a40-9720-0cd173209cd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:07.495762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8140c-9dcf-4a40-9720-0cd173209cd7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.512691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 385/774] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:07 2022] GET /identity/v3/role_assignments?user.id=96655b920bda45cb872624aa4183cb58&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:07.543302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 390/775] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:07.567216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8140c-9dcf-4a40-9720-0cd173209cd7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.696973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8140c-9dcf-4a40-9720-0cd173209cd7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.732596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086d93b-72a2-4b12-973d-6da620d3bc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.733675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086d93b-72a2-4b12-973d-6da620d3bc4d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.733968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086d93b-72a2-4b12-973d-6da620d3bc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.734246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086d93b-72a2-4b12-973d-6da620d3bc4d None admin] PATH_INFO: `/v3/services/network` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.734913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086d93b-72a2-4b12-973d-6da620d3bc4d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:07.736353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086d93b-72a2-4b12-973d-6da620d3bc4d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a086d93b-72a2-4b12-973d-6da620d3bc4d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:07.749576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:07.750597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:07.751451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 386/776] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:07 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:07.756962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2fa95-8451-42d4-bf7d-add8b344ad66 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.875931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2fa95-8451-42d4-bf7d-add8b344ad66 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.919964 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15ed19-8610-4a06-912e-6af817e7d09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.921015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15ed19-8610-4a06-912e-6af817e7d09a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.921498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15ed19-8610-4a06-912e-6af817e7d09a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.921972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15ed19-8610-4a06-912e-6af817e7d09a None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.922898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15ed19-8610-4a06-912e-6af817e7d09a None admin] RBAC: Adding query filter params (name=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:07.923401 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15ed19-8610-4a06-912e-6af817e7d09a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:07.924995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15ed19-8610-4a06-912e-6af817e7d09a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.926273 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:07.933939 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 391/777] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:07.939996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086d93b-72a2-4b12-973d-6da620d3bc4d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.013760 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:08.083869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086d93b-72a2-4b12-973d-6da620d3bc4d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.139407 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ea2ea-2293-4801-af3d-f7e42e35dcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.140046 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ea2ea-2293-4801-af3d-f7e42e35dcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.140333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ea2ea-2293-4801-af3d-f7e42e35dcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.140612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ea2ea-2293-4801-af3d-f7e42e35dcc0 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.141328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ea2ea-2293-4801-af3d-f7e42e35dcc0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:08.141639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ea2ea-2293-4801-af3d-f7e42e35dcc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:08.143008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ea2ea-2293-4801-af3d-f7e42e35dcc0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:08.151884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 387/778] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:08.158092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15ed19-8610-4a06-912e-6af817e7d09a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.295084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15ed19-8610-4a06-912e-6af817e7d09a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.329465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b4b00-35f9-4817-a3f3-2d38c51119e9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.329975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b4b00-35f9-4817-a3f3-2d38c51119e9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.330052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b4b00-35f9-4817-a3f3-2d38c51119e9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.331640 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 388/779] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:08.336812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6931b-d015-450f-8987-151198255b48 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.337210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6931b-d015-450f-8987-151198255b48 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.337500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6931b-d015-450f-8987-151198255b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.338277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ccbbb-68d3-4d00-be30-0684107631ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.338946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ccbbb-68d3-4d00-be30-0684107631ad None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.339225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ccbbb-68d3-4d00-be30-0684107631ad None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.339500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ccbbb-68d3-4d00-be30-0684107631ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.340318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ccbbb-68d3-4d00-be30-0684107631ad None admin] RBAC: Adding query filter params (interface=public, service_id=5ca72b3368144749a017b6219d3a4b7a, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:08.340632 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ccbbb-68d3-4d00-be30-0684107631ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:08.342548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ccbbb-68d3-4d00-be30-0684107631ad None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:08.350384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 392/780] 10.222.0.65 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/endpoints?service_id=5ca72b3368144749a017b6219d3a4b7a&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:08.358489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ccbbb-68d3-4d00-be30-0684107631ad None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.360138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d0e6931b-d015-450f-8987-151198255b48 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:08.361474 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d0e6931b-d015-450f-8987-151198255b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:08.456119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:08.459604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ccbbb-68d3-4d00-be30-0684107631ad None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.501360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 389/781] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:08.510679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bada249-5303-4868-b5da-26032393a94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.511373 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bada249-5303-4868-b5da-26032393a94b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.511668 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bada249-5303-4868-b5da-26032393a94b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.511952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bada249-5303-4868-b5da-26032393a94b None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.512647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bada249-5303-4868-b5da-26032393a94b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:08.514115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bada249-5303-4868-b5da-26032393a94b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:08.520416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 393/782] 10.222.0.65 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/services => generated 772 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:08.529241 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6931b-d015-450f-8987-151198255b48 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.575620 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:08.635601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6931b-d015-450f-8987-151198255b48 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.672510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c7c80-52d0-40f4-844e-101357fe0b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.673323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c7c80-52d0-40f4-844e-101357fe0b54 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.673602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c7c80-52d0-40f4-844e-101357fe0b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.673877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c7c80-52d0-40f4-844e-101357fe0b54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.686343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c7c80-52d0-40f4-844e-101357fe0b54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-850c7c80-52d0-40f4-844e-101357fe0b54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:08.694175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:08.695190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:08.696220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 390/783] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:08.703307 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bada249-5303-4868-b5da-26032393a94b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.830571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bada249-5303-4868-b5da-26032393a94b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.858592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:08.868245 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9b20-de0b-4895-ae9f-33ff6679a6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.868929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9b20-de0b-4895-ae9f-33ff6679a6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.869217 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9b20-de0b-4895-ae9f-33ff6679a6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.869497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9b20-de0b-4895-ae9f-33ff6679a6d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.870262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d9b20-de0b-4895-ae9f-33ff6679a6d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:08.870573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d9b20-de0b-4895-ae9f-33ff6679a6d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:08.871976 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d9b20-de0b-4895-ae9f-33ff6679a6d5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:08.882396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 394/784] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:08.888131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c7c80-52d0-40f4-844e-101357fe0b54 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.027139 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c7c80-52d0-40f4-844e-101357fe0b54 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.031571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.077673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b7cb7-8bfe-44e6-a23f-865f225b0ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.078863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b7cb7-8bfe-44e6-a23f-865f225b0ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.079343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b7cb7-8bfe-44e6-a23f-865f225b0ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.079844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b7cb7-8bfe-44e6-a23f-865f225b0ba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.112949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b7cb7-8bfe-44e6-a23f-865f225b0ba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.114781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b7cb7-8bfe-44e6-a23f-865f225b0ba3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-797b7cb7-8bfe-44e6-a23f-865f225b0ba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:09.142835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:09.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:09.144722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:09.144722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:09.144722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:09.144722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:09.151208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 391/785] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:09.156782 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9b20-de0b-4895-ae9f-33ff6679a6d5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.277452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9b20-de0b-4895-ae9f-33ff6679a6d5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.313773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebdcb4-65da-424e-a058-120c0ff4dd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.314458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebdcb4-65da-424e-a058-120c0ff4dd06 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.320023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebdcb4-65da-424e-a058-120c0ff4dd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.320322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebdcb4-65da-424e-a058-120c0ff4dd06 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.321078 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebdcb4-65da-424e-a058-120c0ff4dd06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:09.321398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebdcb4-65da-424e-a058-120c0ff4dd06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.322776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebdcb4-65da-424e-a058-120c0ff4dd06 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:09.335377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 395/786] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:09.341133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b7cb7-8bfe-44e6-a23f-865f225b0ba3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.465381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8b5c9-453c-41cf-b936-a0f705dd2867 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.466364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8b5c9-453c-41cf-b936-a0f705dd2867 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.466842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8b5c9-453c-41cf-b936-a0f705dd2867 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.472846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 396/787] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:09.477956 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966ab21-bd8d-4655-9ff1-23e98863132e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.478526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966ab21-bd8d-4655-9ff1-23e98863132e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.479009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966ab21-bd8d-4655-9ff1-23e98863132e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.499588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b7cb7-8bfe-44e6-a23f-865f225b0ba3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.519533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d966ab21-bd8d-4655-9ff1-23e98863132e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:09.521069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d966ab21-bd8d-4655-9ff1-23e98863132e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:09.539485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae3759-540c-4e3a-8d58-e7a15cea7726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.540312 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae3759-540c-4e3a-8d58-e7a15cea7726 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.540590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae3759-540c-4e3a-8d58-e7a15cea7726 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.540860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae3759-540c-4e3a-8d58-e7a15cea7726 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.550765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae3759-540c-4e3a-8d58-e7a15cea7726 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.552347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae3759-540c-4e3a-8d58-e7a15cea7726 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-abae3759-540c-4e3a-8d58-e7a15cea7726 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:09.576885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:09.578178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:09.579127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:09.579127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:09.579127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:09.579127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:09.579127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:09.579127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:09.579127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:09.579127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:09.579127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:09.579127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:09.579127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 392/788] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:09.582307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae3759-540c-4e3a-8d58-e7a15cea7726 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.602795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.604035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.652599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 397/789] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:09.680771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966ab21-bd8d-4655-9ff1-23e98863132e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.743923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae3759-540c-4e3a-8d58-e7a15cea7726 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.777689 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56e6e2-2933-4fa6-90d1-e7181d61fa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.778364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56e6e2-2933-4fa6-90d1-e7181d61fa4c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.779214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56e6e2-2933-4fa6-90d1-e7181d61fa4c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.779495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56e6e2-2933-4fa6-90d1-e7181d61fa4c None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.780367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56e6e2-2933-4fa6-90d1-e7181d61fa4c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:09.780674 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56e6e2-2933-4fa6-90d1-e7181d61fa4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.782406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56e6e2-2933-4fa6-90d1-e7181d61fa4c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:09.798492 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966ab21-bd8d-4655-9ff1-23e98863132e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.804323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 393/790] 10.222.0.65 () {56 vars in 1135 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:09.808523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06f021-404c-4b34-9b5a-7ae7d5313029 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.809206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06f021-404c-4b34-9b5a-7ae7d5313029 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.809690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06f021-404c-4b34-9b5a-7ae7d5313029 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.812921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 394/791] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:09.815100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06f021-404c-4b34-9b5a-7ae7d5313029 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.843241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdf0eb-c8c9-4ed8-96a6-beaeae984895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.844942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdf0eb-c8c9-4ed8-96a6-beaeae984895 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.845429 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdf0eb-c8c9-4ed8-96a6-beaeae984895 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.846126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdf0eb-c8c9-4ed8-96a6-beaeae984895 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.861302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdf0eb-c8c9-4ed8-96a6-beaeae984895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2dcdf0eb-c8c9-4ed8-96a6-beaeae984895 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:09.871031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:09.872489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:09.873339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:09.873339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:09.873339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:09.873339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:09.873339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:09.873339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:09.873339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:09.873339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:09.873339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:09.873339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:09.873339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:09.873339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:09.876411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 398/792] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:09.878418 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5e7e8-110e-4d41-97b6-e13ddd2e01f6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.878984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5e7e8-110e-4d41-97b6-e13ddd2e01f6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.879438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5e7e8-110e-4d41-97b6-e13ddd2e01f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.900989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-60e5e7e8-110e-4d41-97b6-e13ddd2e01f6 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:09.902694 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-60e5e7e8-110e-4d41-97b6-e13ddd2e01f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:09.904470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.941735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.945282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06f021-404c-4b34-9b5a-7ae7d5313029 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.993453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de82e32-088e-4385-ae7a-cec02c6f4342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.994240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 399/793] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:09.994812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de82e32-088e-4385-ae7a-cec02c6f4342 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.995044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5e7e8-110e-4d41-97b6-e13ddd2e01f6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.995443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de82e32-088e-4385-ae7a-cec02c6f4342 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.995887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de82e32-088e-4385-ae7a-cec02c6f4342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.009785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de82e32-088e-4385-ae7a-cec02c6f4342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.012221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de82e32-088e-4385-ae7a-cec02c6f4342 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5de82e32-088e-4385-ae7a-cec02c6f4342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:10.037194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:10.038202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:10.039107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:10.039107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:10.039107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:10.039107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:10.041933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 395/794] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:10.043555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de82e32-088e-4385-ae7a-cec02c6f4342 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.118892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5e7e8-110e-4d41-97b6-e13ddd2e01f6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.154428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eaf0a9-d246-486a-b7f2-fa7b6575a126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.155094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaf0a9-d246-486a-b7f2-fa7b6575a126 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.155376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaf0a9-d246-486a-b7f2-fa7b6575a126 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.155676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaf0a9-d246-486a-b7f2-fa7b6575a126 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.156413 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eaf0a9-d246-486a-b7f2-fa7b6575a126 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:10.156717 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eaf0a9-d246-486a-b7f2-fa7b6575a126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.158082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eaf0a9-d246-486a-b7f2-fa7b6575a126 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.159138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de82e32-088e-4385-ae7a-cec02c6f4342 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.170564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 400/795] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:10.170564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eaf0a9-d246-486a-b7f2-fa7b6575a126 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.188390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.198391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.202484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416bd48-d568-4a2b-8447-3b9a2e90a944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.203303 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416bd48-d568-4a2b-8447-3b9a2e90a944 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.203635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416bd48-d568-4a2b-8447-3b9a2e90a944 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.203948 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416bd48-d568-4a2b-8447-3b9a2e90a944 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.204751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416bd48-d568-4a2b-8447-3b9a2e90a944 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.206125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416bd48-d568-4a2b-8447-3b9a2e90a944 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3416bd48-d568-4a2b-8447-3b9a2e90a944 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:10.219073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:10.220228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:10.221222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 396/796] 10.222.0.65 () {56 vars in 1129 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:10.222299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1f3bc-6f44-4612-bf3b-a1da05fe881f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.222299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1f3bc-6f44-4612-bf3b-a1da05fe881f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.222299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1f3bc-6f44-4612-bf3b-a1da05fe881f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.223416 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 397/797] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:10.231861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1f3bc-6f44-4612-bf3b-a1da05fe881f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.276794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eaf0a9-d246-486a-b7f2-fa7b6575a126 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.312385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a935d-4f0e-4dea-962e-a3751a3fc373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.313052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a935d-4f0e-4dea-962e-a3751a3fc373 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.313341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a935d-4f0e-4dea-962e-a3751a3fc373 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.313622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a935d-4f0e-4dea-962e-a3751a3fc373 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.314343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a935d-4f0e-4dea-962e-a3751a3fc373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:10.315297 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a935d-4f0e-4dea-962e-a3751a3fc373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.316059 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a935d-4f0e-4dea-962e-a3751a3fc373 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.328166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 401/798] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:10.328775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a935d-4f0e-4dea-962e-a3751a3fc373 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.350665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1f3bc-6f44-4612-bf3b-a1da05fe881f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.385604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842d354-34e8-4c07-b2bc-ac0d41f97d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.386386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842d354-34e8-4c07-b2bc-ac0d41f97d22 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.386664 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842d354-34e8-4c07-b2bc-ac0d41f97d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.386943 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842d354-34e8-4c07-b2bc-ac0d41f97d22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.397233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842d354-34e8-4c07-b2bc-ac0d41f97d22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.398666 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842d354-34e8-4c07-b2bc-ac0d41f97d22 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0842d354-34e8-4c07-b2bc-ac0d41f97d22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:10.416969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:10.417897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:10.418789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:10.418789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:10.418789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:10.418789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:10.418789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 398/799] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:10.420129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3089c2-bf1b-44f9-955d-261672e2f2c2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.420242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3089c2-bf1b-44f9-955d-261672e2f2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.420504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3089c2-bf1b-44f9-955d-261672e2f2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.429906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.437380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ed3089c2-bf1b-44f9-955d-261672e2f2c2 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:10.439020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ed3089c2-bf1b-44f9-955d-261672e2f2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:10.441296 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a935d-4f0e-4dea-962e-a3751a3fc373 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.474837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.476264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fe942-cbd1-4903-a8cb-2062d9a3bad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.476942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fe942-cbd1-4903-a8cb-2062d9a3bad9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.477233 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fe942-cbd1-4903-a8cb-2062d9a3bad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.477604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fe942-cbd1-4903-a8cb-2062d9a3bad9 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.478430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fe942-cbd1-4903-a8cb-2062d9a3bad9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:10.478733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fe942-cbd1-4903-a8cb-2062d9a3bad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.480153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fe942-cbd1-4903-a8cb-2062d9a3bad9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.485682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 402/800] 10.222.0.65 () {56 vars in 1124 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:10.486480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fe942-cbd1-4903-a8cb-2062d9a3bad9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.517718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 399/801] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:10.518402 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3089c2-bf1b-44f9-955d-261672e2f2c2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.617166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fe942-cbd1-4903-a8cb-2062d9a3bad9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.620273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3089c2-bf1b-44f9-955d-261672e2f2c2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.655567 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029061a-f107-4937-905a-894d62ef9476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.655940 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81f441-7f48-48b8-848b-81d64b51c220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.656395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029061a-f107-4937-905a-894d62ef9476 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.656525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81f441-7f48-48b8-848b-81d64b51c220 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.656680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029061a-f107-4937-905a-894d62ef9476 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.656831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81f441-7f48-48b8-848b-81d64b51c220 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.656980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029061a-f107-4937-905a-894d62ef9476 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.657069 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81f441-7f48-48b8-848b-81d64b51c220 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.657774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81f441-7f48-48b8-848b-81d64b51c220 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:10.658083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81f441-7f48-48b8-848b-81d64b51c220 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.659471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81f441-7f48-48b8-848b-81d64b51c220 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.664096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029061a-f107-4937-905a-894d62ef9476 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.665526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029061a-f107-4937-905a-894d62ef9476 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.675251 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 400/802] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:10.675251 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81f441-7f48-48b8-848b-81d64b51c220 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d029061a-f107-4937-905a-894d62ef9476 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:10.684284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:10.685392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:10.686322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:10.686322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:10.686322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:10.686322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:10.686322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:10.686322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:10.686322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:10.686322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:10.686322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 403/803] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:10.686799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029061a-f107-4937-905a-894d62ef9476 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.703060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.712319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.790411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029061a-f107-4937-905a-894d62ef9476 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.793093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81f441-7f48-48b8-848b-81d64b51c220 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.823605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b3190-e322-41ec-b73a-3d849964d4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.824434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b3190-e322-41ec-b73a-3d849964d4ed None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.824725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b3190-e322-41ec-b73a-3d849964d4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.825005 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b3190-e322-41ec-b73a-3d849964d4ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.828679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f3bc8-0bc9-446c-80ee-58cd4f8521db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.829345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f3bc8-0bc9-446c-80ee-58cd4f8521db None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.829624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f3bc8-0bc9-446c-80ee-58cd4f8521db None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.829894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f3bc8-0bc9-446c-80ee-58cd4f8521db None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.830602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f3bc8-0bc9-446c-80ee-58cd4f8521db None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:10.830902 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f3bc8-0bc9-446c-80ee-58cd4f8521db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.832454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f3bc8-0bc9-446c-80ee-58cd4f8521db None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.836239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b3190-e322-41ec-b73a-3d849964d4ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:10.842680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 401/804] 10.222.0.65 () {56 vars in 1124 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:10.842680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f3bc8-0bc9-446c-80ee-58cd4f8521db None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-658b3190-e322-41ec-b73a-3d849964d4ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:10.844561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:10.845452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:10.846320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 404/805] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:10.847564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b3190-e322-41ec-b73a-3d849964d4ed None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.930091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.945767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.953778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f3bc8-0bc9-446c-80ee-58cd4f8521db None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.958545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b3190-e322-41ec-b73a-3d849964d4ed None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.991360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc94f5f-bdb8-4822-ba49-889a83ccc05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.992184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc94f5f-bdb8-4822-ba49-889a83ccc05c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.992465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc94f5f-bdb8-4822-ba49-889a83ccc05c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.992737 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc94f5f-bdb8-4822-ba49-889a83ccc05c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.995793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee0ab8-40cf-481b-ac3a-dddb2f7a4f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.996451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee0ab8-40cf-481b-ac3a-dddb2f7a4f91 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.996732 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee0ab8-40cf-481b-ac3a-dddb2f7a4f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.997009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee0ab8-40cf-481b-ac3a-dddb2f7a4f91 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.998301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee0ab8-40cf-481b-ac3a-dddb2f7a4f91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:10.998606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee0ab8-40cf-481b-ac3a-dddb2f7a4f91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.000050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee0ab8-40cf-481b-ac3a-dddb2f7a4f91 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.000280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc94f5f-bdb8-4822-ba49-889a83ccc05c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.001685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc94f5f-bdb8-4822-ba49-889a83ccc05c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.012258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 405/806] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:11.013702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee0ab8-40cf-481b-ac3a-dddb2f7a4f91 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3cc94f5f-bdb8-4822-ba49-889a83ccc05c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:11.016231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:11.017176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:11.018128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:11.018128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:11.018128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:11.018128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:11.018128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:11.018128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:11.018128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:11.018128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:11.018128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:11.018128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:11.018128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 402/807] 10.222.0.65 () {56 vars in 1093 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:11.018128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc94f5f-bdb8-4822-ba49-889a83ccc05c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.114952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee0ab8-40cf-481b-ac3a-dddb2f7a4f91 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.115112 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc94f5f-bdb8-4822-ba49-889a83ccc05c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.148426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e37b9-3650-4daa-9606-35fbc4aabee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.149081 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e37b9-3650-4daa-9606-35fbc4aabee8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.149273 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec139de-e446-400a-90ba-38714df89207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.149384 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e37b9-3650-4daa-9606-35fbc4aabee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.149612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e37b9-3650-4daa-9606-35fbc4aabee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.149932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec139de-e446-400a-90ba-38714df89207 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.150220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec139de-e446-400a-90ba-38714df89207 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.150349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e37b9-3650-4daa-9606-35fbc4aabee8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:11.150501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec139de-e446-400a-90ba-38714df89207 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.150653 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e37b9-3650-4daa-9606-35fbc4aabee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.151242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec139de-e446-400a-90ba-38714df89207 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.152034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e37b9-3650-4daa-9606-35fbc4aabee8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.152636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec139de-e446-400a-90ba-38714df89207 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.156195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.160402 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 403/808] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:11.161699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a0680-4b85-4d05-b3bb-f2227586e486 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.161993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a0680-4b85-4d05-b3bb-f2227586e486 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.162266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a0680-4b85-4d05-b3bb-f2227586e486 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.163480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 404/809] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:11.164174 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a0680-4b85-4d05-b3bb-f2227586e486 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.177554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 406/810] 10.222.0.65 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:11.178377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec139de-e446-400a-90ba-38714df89207 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.195731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.298040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec139de-e446-400a-90ba-38714df89207 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.298545 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a0680-4b85-4d05-b3bb-f2227586e486 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.339400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ae82e-805d-4499-bc00-b82eb95aec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.339821 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ae82e-805d-4499-bc00-b82eb95aec88 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.340100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ae82e-805d-4499-bc00-b82eb95aec88 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.340373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ae82e-805d-4499-bc00-b82eb95aec88 None admin] PATH_INFO: `/v3/projects/8458544c021049c587856343e8001429/users/d3d8f8acd0484857b451877baf03e895/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.340993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fec9ca-2fa1-4a7d-a69a-0166506c1b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.341705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fec9ca-2fa1-4a7d-a69a-0166506c1b5b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.341993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fec9ca-2fa1-4a7d-a69a-0166506c1b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.342274 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fec9ca-2fa1-4a7d-a69a-0166506c1b5b None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.343122 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fec9ca-2fa1-4a7d-a69a-0166506c1b5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:11.343437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fec9ca-2fa1-4a7d-a69a-0166506c1b5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.343542 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ae82e-805d-4499-bc00-b82eb95aec88 None admin] RBAC: Authorizing `identity:create_grant(project_id=8458544c021049c587856343e8001429, user_id=d3d8f8acd0484857b451877baf03e895, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.344972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fec9ca-2fa1-4a7d-a69a-0166506c1b5b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.345085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ae82e-805d-4499-bc00-b82eb95aec88 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.364406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 405/811] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:11 2022] PUT /identity/v3/projects/8458544c021049c587856343e8001429/users/d3d8f8acd0484857b451877baf03e895/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:11.365282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ae82e-805d-4499-bc00-b82eb95aec88 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.367814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 407/812] 10.222.0.65 () {56 vars in 1133 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:11.369853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d693c-2cfe-485d-a192-85c0747f42c6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.370329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d693c-2cfe-485d-a192-85c0747f42c6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.370826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d693c-2cfe-485d-a192-85c0747f42c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.389192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-899d693c-2cfe-485d-a192-85c0747f42c6 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:11.390438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-899d693c-2cfe-485d-a192-85c0747f42c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:11.443745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.465042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.481314 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 408/813] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:11.484229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d693c-2cfe-485d-a192-85c0747f42c6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.529771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ae82e-805d-4499-bc00-b82eb95aec88 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.571578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5408e-3c52-43c0-b05a-e92cbb9ef44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.572411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5408e-3c52-43c0-b05a-e92cbb9ef44b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.572718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5408e-3c52-43c0-b05a-e92cbb9ef44b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.572981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5408e-3c52-43c0-b05a-e92cbb9ef44b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.584238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5408e-3c52-43c0-b05a-e92cbb9ef44b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.585694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5408e-3c52-43c0-b05a-e92cbb9ef44b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3fc5408e-3c52-43c0-b05a-e92cbb9ef44b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:11.604594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:11.605537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:11.606419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:11.606419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:11.606419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:11.606419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:11.606419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 406/814] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:11.607694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5408e-3c52-43c0-b05a-e92cbb9ef44b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.625932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d693c-2cfe-485d-a192-85c0747f42c6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.663223 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0d924-3688-42e6-be10-72cee11f6919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.664304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0d924-3688-42e6-be10-72cee11f6919 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.664765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0d924-3688-42e6-be10-72cee11f6919 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.665211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0d924-3688-42e6-be10-72cee11f6919 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.715203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5408e-3c52-43c0-b05a-e92cbb9ef44b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.717553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.740856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.749078 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0d924-3688-42e6-be10-72cee11f6919 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.751355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0d924-3688-42e6-be10-72cee11f6919 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.765016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a18438-12ba-428b-bb16-98477c39c080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.766582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a18438-12ba-428b-bb16-98477c39c080 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.767343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a18438-12ba-428b-bb16-98477c39c080 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.767854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a18438-12ba-428b-bb16-98477c39c080 None admin] PATH_INFO: `/v3/services/network` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.768966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a18438-12ba-428b-bb16-98477c39c080 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.771288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a18438-12ba-428b-bb16-98477c39c080 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-97a18438-12ba-428b-bb16-98477c39c080 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:11.787937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:11.788945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:11.789782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:11.798077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 407/815] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:11.799204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a18438-12ba-428b-bb16-98477c39c080 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.864513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 409/816] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:11.865945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0d924-3688-42e6-be10-72cee11f6919 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.935774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a18438-12ba-428b-bb16-98477c39c080 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.952066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.972622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0d924-3688-42e6-be10-72cee11f6919 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.981519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be39137-7afd-471f-bf40-1f0a9bc57733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.983195 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be39137-7afd-471f-bf40-1f0a9bc57733 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.983625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be39137-7afd-471f-bf40-1f0a9bc57733 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.984090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be39137-7afd-471f-bf40-1f0a9bc57733 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.985425 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be39137-7afd-471f-bf40-1f0a9bc57733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:11.985882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be39137-7afd-471f-bf40-1f0a9bc57733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.988073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be39137-7afd-471f-bf40-1f0a9bc57733 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.009896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 408/817] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:12.010677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be39137-7afd-471f-bf40-1f0a9bc57733 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.016640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1db32-bae3-4809-912d-485aceb7adbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.017644 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1db32-bae3-4809-912d-485aceb7adbb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.018073 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1db32-bae3-4809-912d-485aceb7adbb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.018494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1db32-bae3-4809-912d-485aceb7adbb None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.019358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1db32-bae3-4809-912d-485aceb7adbb None admin] RBAC: Adding query filter params (name=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:12.019842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1db32-bae3-4809-912d-485aceb7adbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.021878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1db32-bae3-4809-912d-485aceb7adbb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.029146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 410/818] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:12.030107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1db32-bae3-4809-912d-485aceb7adbb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.067030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.128652 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be39137-7afd-471f-bf40-1f0a9bc57733 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.150417 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1db32-bae3-4809-912d-485aceb7adbb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.163625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8cc3f-ac6f-4301-8c6d-14488a0ce5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.164768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8cc3f-ac6f-4301-8c6d-14488a0ce5ff None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.165231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8cc3f-ac6f-4301-8c6d-14488a0ce5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.165714 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8cc3f-ac6f-4301-8c6d-14488a0ce5ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.177567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8cc3f-ac6f-4301-8c6d-14488a0ce5ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.179454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8cc3f-ac6f-4301-8c6d-14488a0ce5ff None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.187410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76385c97-e9aa-406b-9fea-edb4b65e6b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.188374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76385c97-e9aa-406b-9fea-edb4b65e6b88 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.188761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76385c97-e9aa-406b-9fea-edb4b65e6b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.189106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76385c97-e9aa-406b-9fea-edb4b65e6b88 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.196845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76385c97-e9aa-406b-9fea-edb4b65e6b88 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.198407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76385c97-e9aa-406b-9fea-edb4b65e6b88 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.200428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-95e8cc3f-ac6f-4301-8c6d-14488a0ce5ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:12.203988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:12.204869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:12.205756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:12.205756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:12.205756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:12.205756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:12.208583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 409/819] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:12.209507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8cc3f-ac6f-4301-8c6d-14488a0ce5ff None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-76385c97-e9aa-406b-9fea-edb4b65e6b88 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:12.216555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:12.217724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:12.218972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:12.218972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:12.218972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:12.218972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:12.218972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:12.218972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:12.218972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:12.218972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:12.218972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:12.218972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:12.223967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 411/820] 10.222.0.65 () {56 vars in 1117 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:12.224811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76385c97-e9aa-406b-9fea-edb4b65e6b88 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.341531 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.343847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76385c97-e9aa-406b-9fea-edb4b65e6b88 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.344646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8cc3f-ac6f-4301-8c6d-14488a0ce5ff None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.383545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b5aaf-6969-49f1-b0e8-f8b5fe3be14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.384241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b5aaf-6969-49f1-b0e8-f8b5fe3be14c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.384521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b5aaf-6969-49f1-b0e8-f8b5fe3be14c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.384795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b5aaf-6969-49f1-b0e8-f8b5fe3be14c None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.385515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b5aaf-6969-49f1-b0e8-f8b5fe3be14c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:12.385821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b5aaf-6969-49f1-b0e8-f8b5fe3be14c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.387246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b5aaf-6969-49f1-b0e8-f8b5fe3be14c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.396415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e20a9-3e01-4dd4-af8c-1f29a94211ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.397907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e20a9-3e01-4dd4-af8c-1f29a94211ef None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.398369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e20a9-3e01-4dd4-af8c-1f29a94211ef None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.398819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e20a9-3e01-4dd4-af8c-1f29a94211ef None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.399726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e20a9-3e01-4dd4-af8c-1f29a94211ef None admin] RBAC: Adding query filter params (type=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:12.400236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e20a9-3e01-4dd4-af8c-1f29a94211ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.401770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e20a9-3e01-4dd4-af8c-1f29a94211ef None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.403867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 412/821] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:12.405027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b5aaf-6969-49f1-b0e8-f8b5fe3be14c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.408258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 410/822] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:12.410156 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e20a9-3e01-4dd4-af8c-1f29a94211ef None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.482830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.518420 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e20a9-3e01-4dd4-af8c-1f29a94211ef None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.536819 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b5aaf-6969-49f1-b0e8-f8b5fe3be14c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.561237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d085b-d701-4b60-8cfc-73f0d7f9e803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.562166 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d085b-d701-4b60-8cfc-73f0d7f9e803 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.562509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d085b-d701-4b60-8cfc-73f0d7f9e803 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.562855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d085b-d701-4b60-8cfc-73f0d7f9e803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.571953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d085b-d701-4b60-8cfc-73f0d7f9e803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.573960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d085b-d701-4b60-8cfc-73f0d7f9e803 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.579023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73b00c-ef3c-47fd-a62e-51d3890bb338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.580003 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73b00c-ef3c-47fd-a62e-51d3890bb338 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.580649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73b00c-ef3c-47fd-a62e-51d3890bb338 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.581342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73b00c-ef3c-47fd-a62e-51d3890bb338 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.582389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73b00c-ef3c-47fd-a62e-51d3890bb338 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:12.582872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73b00c-ef3c-47fd-a62e-51d3890bb338 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.584540 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73b00c-ef3c-47fd-a62e-51d3890bb338 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-912d085b-d701-4b60-8cfc-73f0d7f9e803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:12.589968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:12.590823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:12.592322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:12.592322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:12.592322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:12.592322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:12.592322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:12.592322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:12.592322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:12.592322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:12.595762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 411/823] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:12.596731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d085b-d701-4b60-8cfc-73f0d7f9e803 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.613531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 413/824] 10.222.0.65 () {56 vars in 1149 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:12.614973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73b00c-ef3c-47fd-a62e-51d3890bb338 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.628282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.715198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d085b-d701-4b60-8cfc-73f0d7f9e803 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.729143 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73b00c-ef3c-47fd-a62e-51d3890bb338 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.747554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.756000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c670c55-a8c3-4cd1-8e95-04284f661324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.756902 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c670c55-a8c3-4cd1-8e95-04284f661324 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.757370 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c670c55-a8c3-4cd1-8e95-04284f661324 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.757901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c670c55-a8c3-4cd1-8e95-04284f661324 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.758967 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c670c55-a8c3-4cd1-8e95-04284f661324 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.760565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c670c55-a8c3-4cd1-8e95-04284f661324 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.766544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633e90c-8d91-451d-93d6-b29b8811c63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.767302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633e90c-8d91-451d-93d6-b29b8811c63c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.767669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633e90c-8d91-451d-93d6-b29b8811c63c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.768022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633e90c-8d91-451d-93d6-b29b8811c63c None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.768878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633e90c-8d91-451d-93d6-b29b8811c63c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:12.769254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633e90c-8d91-451d-93d6-b29b8811c63c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.770836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633e90c-8d91-451d-93d6-b29b8811c63c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.784464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 412/825] 10.222.0.65 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:12.785632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c670c55-a8c3-4cd1-8e95-04284f661324 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.790251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 414/826] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:12.791485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633e90c-8d91-451d-93d6-b29b8811c63c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.877341 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.907698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c670c55-a8c3-4cd1-8e95-04284f661324 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.955193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078c9f2-41c7-4850-b516-9eac9e1d0fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.956543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078c9f2-41c7-4850-b516-9eac9e1d0fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.958093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078c9f2-41c7-4850-b516-9eac9e1d0fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.958589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078c9f2-41c7-4850-b516-9eac9e1d0fb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.981939 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078c9f2-41c7-4850-b516-9eac9e1d0fb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.984742 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078c9f2-41c7-4850-b516-9eac9e1d0fb8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.001070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633e90c-8d91-451d-93d6-b29b8811c63c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f078c9f2-41c7-4850-b516-9eac9e1d0fb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:13.021617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:13.022578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:13.023462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:13.023462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:13.023462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:13.023462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:13.025946 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 413/827] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:13.032268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078c9f2-41c7-4850-b516-9eac9e1d0fb8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.066916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903333d8-5a82-4381-9d7a-5422b7a18592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.067726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903333d8-5a82-4381-9d7a-5422b7a18592 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.068015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903333d8-5a82-4381-9d7a-5422b7a18592 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.068376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903333d8-5a82-4381-9d7a-5422b7a18592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.069126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903333d8-5a82-4381-9d7a-5422b7a18592 None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=8458544c021049c587856343e8001429, user.id=b01d875111bd43698bcf99d30f7a7f9a) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:13.069447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903333d8-5a82-4381-9d7a-5422b7a18592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.071115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903333d8-5a82-4381-9d7a-5422b7a18592 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.079098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 415/828] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/role_assignments?user.id=b01d875111bd43698bcf99d30f7a7f9a&scope.project.id=8458544c021049c587856343e8001429&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:13.159641 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078c9f2-41c7-4850-b516-9eac9e1d0fb8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.191986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.201896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54e75c-817a-4d7e-aec8-8dc75770cd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.202952 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54e75c-817a-4d7e-aec8-8dc75770cd33 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.203443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54e75c-817a-4d7e-aec8-8dc75770cd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.203954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54e75c-817a-4d7e-aec8-8dc75770cd33 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.204899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54e75c-817a-4d7e-aec8-8dc75770cd33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:13.205415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54e75c-817a-4d7e-aec8-8dc75770cd33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.207036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54e75c-817a-4d7e-aec8-8dc75770cd33 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.225705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 414/829] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:13.230940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903333d8-5a82-4381-9d7a-5422b7a18592 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.283162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.349209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc99dbe-8999-4548-9c32-253a6d884b05 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.349552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc99dbe-8999-4548-9c32-253a6d884b05 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.349780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc99dbe-8999-4548-9c32-253a6d884b05 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.350256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903333d8-5a82-4381-9d7a-5422b7a18592 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.351328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 415/830] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:13.356313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c38a79-1b81-4152-985d-d1cf6c49597e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.356848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c38a79-1b81-4152-985d-d1cf6c49597e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.357350 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c38a79-1b81-4152-985d-d1cf6c49597e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.381042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-03c38a79-1b81-4152-985d-d1cf6c49597e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:13.382338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-03c38a79-1b81-4152-985d-d1cf6c49597e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:13.394229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf443b-8bd1-4871-ac4b-1a9d58e8f128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.395391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf443b-8bd1-4871-ac4b-1a9d58e8f128 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.395907 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf443b-8bd1-4871-ac4b-1a9d58e8f128 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.396398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf443b-8bd1-4871-ac4b-1a9d58e8f128 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.404085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf443b-8bd1-4871-ac4b-1a9d58e8f128 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.405713 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf443b-8bd1-4871-ac4b-1a9d58e8f128 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1cbf443b-8bd1-4871-ac4b-1a9d58e8f128 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:13.420346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:13.421356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:13.423231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:13.423231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:13.423231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:13.423231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:13.423231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:13.423231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:13.423231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:13.423231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:13.423231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 416/831] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:13.427032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf443b-8bd1-4871-ac4b-1a9d58e8f128 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.468577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 416/832] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:13.492616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c38a79-1b81-4152-985d-d1cf6c49597e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.557504 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.560985 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf443b-8bd1-4871-ac4b-1a9d58e8f128 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.596983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391eb987-76d1-4e5c-9057-185f9a6db50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.597747 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391eb987-76d1-4e5c-9057-185f9a6db50d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.597929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391eb987-76d1-4e5c-9057-185f9a6db50d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.598209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391eb987-76d1-4e5c-9057-185f9a6db50d None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.599060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391eb987-76d1-4e5c-9057-185f9a6db50d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:13.599319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391eb987-76d1-4e5c-9057-185f9a6db50d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.600799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391eb987-76d1-4e5c-9057-185f9a6db50d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.603921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c38a79-1b81-4152-985d-d1cf6c49597e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.613674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 417/833] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:13.614860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5c1f0-5d9e-470f-b869-a16e302cf854 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.615153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5c1f0-5d9e-470f-b869-a16e302cf854 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.615436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5c1f0-5d9e-470f-b869-a16e302cf854 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.619692 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 418/834] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:13.619692 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5c1f0-5d9e-470f-b869-a16e302cf854 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.630216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.652087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1c4d2-3bd1-4375-8610-02128ad82958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.652896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1c4d2-3bd1-4375-8610-02128ad82958 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.653170 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1c4d2-3bd1-4375-8610-02128ad82958 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.653470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1c4d2-3bd1-4375-8610-02128ad82958 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.654133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1c4d2-3bd1-4375-8610-02128ad82958 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.655497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1c4d2-3bd1-4375-8610-02128ad82958 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1bc1c4d2-3bd1-4375-8610-02128ad82958 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:13.667942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:13.668898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:13.670237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 417/835] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:13.671051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce75d42-1886-41bf-b40d-f6ef6333305e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.671121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce75d42-1886-41bf-b40d-f6ef6333305e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.671867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce75d42-1886-41bf-b40d-f6ef6333305e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.689279 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6ce75d42-1886-41bf-b40d-f6ef6333305e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:13.690460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6ce75d42-1886-41bf-b40d-f6ef6333305e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:13.731044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5c1f0-5d9e-470f-b869-a16e302cf854 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.763691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 418/836] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:13.763691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce75d42-1886-41bf-b40d-f6ef6333305e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.773840 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748b539-21c0-4bf7-be2a-01dab71b01f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.775489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748b539-21c0-4bf7-be2a-01dab71b01f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.776020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748b539-21c0-4bf7-be2a-01dab71b01f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.776886 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748b539-21c0-4bf7-be2a-01dab71b01f4 None admin] PATH_INFO: `/v3/projects/8458544c021049c587856343e8001429/users/96655b920bda45cb872624aa4183cb58/roles/b29c780ef8924837b89e1f06648a9563` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.795710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748b539-21c0-4bf7-be2a-01dab71b01f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8458544c021049c587856343e8001429, user_id=96655b920bda45cb872624aa4183cb58, role_id=b29c780ef8924837b89e1f06648a9563)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.797168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748b539-21c0-4bf7-be2a-01dab71b01f4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.819777 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 419/837] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:13 2022] PUT /identity/v3/projects/8458544c021049c587856343e8001429/users/96655b920bda45cb872624aa4183cb58/roles/b29c780ef8924837b89e1f06648a9563 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:50:13.819777 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748b539-21c0-4bf7-be2a-01dab71b01f4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.834961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.904641 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce75d42-1886-41bf-b40d-f6ef6333305e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.940376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.952618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748b539-21c0-4bf7-be2a-01dab71b01f4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.956232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5cb44-9237-4a54-8839-569c9eb078b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.956953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5cb44-9237-4a54-8839-569c9eb078b7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.957260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5cb44-9237-4a54-8839-569c9eb078b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.957533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5cb44-9237-4a54-8839-569c9eb078b7 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.958254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5cb44-9237-4a54-8839-569c9eb078b7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:13.958561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5cb44-9237-4a54-8839-569c9eb078b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.960008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5cb44-9237-4a54-8839-569c9eb078b7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.965475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 419/838] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:13.970737 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5cb44-9237-4a54-8839-569c9eb078b7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.010578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef317f8-3dc3-423e-9c16-7094e70ae094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.012065 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef317f8-3dc3-423e-9c16-7094e70ae094 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.012357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef317f8-3dc3-423e-9c16-7094e70ae094 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.012777 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef317f8-3dc3-423e-9c16-7094e70ae094 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.026715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef317f8-3dc3-423e-9c16-7094e70ae094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-cef317f8-3dc3-423e-9c16-7094e70ae094 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:14.035531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:14.036568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:14.037446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:14.037446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:14.037446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:14.037446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:14.037446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:14.037446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:14.037446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:14.037446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:14.037446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:14.037446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:14.037446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:14.037446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:14.041978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 420/839] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:14.042854 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef317f8-3dc3-423e-9c16-7094e70ae094 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.101855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5cb44-9237-4a54-8839-569c9eb078b7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.117405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:14.141366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4e83b-9e11-42e1-8e7c-707fe753ed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.142430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4e83b-9e11-42e1-8e7c-707fe753ed32 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.142917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4e83b-9e11-42e1-8e7c-707fe753ed32 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.143404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4e83b-9e11-42e1-8e7c-707fe753ed32 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.144444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4e83b-9e11-42e1-8e7c-707fe753ed32 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:14.145025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4e83b-9e11-42e1-8e7c-707fe753ed32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.146700 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4e83b-9e11-42e1-8e7c-707fe753ed32 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.150423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef317f8-3dc3-423e-9c16-7094e70ae094 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.152519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 420/840] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:14.189032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76161-f33e-40ae-b2f5-ef24a16e4464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.189726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76161-f33e-40ae-b2f5-ef24a16e4464 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.190010 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76161-f33e-40ae-b2f5-ef24a16e4464 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.190288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76161-f33e-40ae-b2f5-ef24a16e4464 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.191050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76161-f33e-40ae-b2f5-ef24a16e4464 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:14.191357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76161-f33e-40ae-b2f5-ef24a16e4464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.192790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76161-f33e-40ae-b2f5-ef24a16e4464 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.202643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 421/841] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:14.208373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4e83b-9e11-42e1-8e7c-707fe753ed32 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.263016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:14.328712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4e83b-9e11-42e1-8e7c-707fe753ed32 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.367332 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7204ab-2f57-4f3c-9ba2-39d34303cae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.368131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7204ab-2f57-4f3c-9ba2-39d34303cae8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.368413 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7204ab-2f57-4f3c-9ba2-39d34303cae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.368693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7204ab-2f57-4f3c-9ba2-39d34303cae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.379677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7204ab-2f57-4f3c-9ba2-39d34303cae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.381130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7204ab-2f57-4f3c-9ba2-39d34303cae8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ed7204ab-2f57-4f3c-9ba2-39d34303cae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:14.403304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:14.404941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:14.405822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:14.405822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:14.405822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:14.405822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:14.409242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 421/842] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:14.414969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76161-f33e-40ae-b2f5-ef24a16e4464 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.517117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76161-f33e-40ae-b2f5-ef24a16e4464 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.547199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:14.552355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03436b3d-8957-4872-a015-c3fe53badeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.553323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03436b3d-8957-4872-a015-c3fe53badeca None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.553832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03436b3d-8957-4872-a015-c3fe53badeca None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.554112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03436b3d-8957-4872-a015-c3fe53badeca None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.555032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03436b3d-8957-4872-a015-c3fe53badeca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:14.555373 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03436b3d-8957-4872-a015-c3fe53badeca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.557186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03436b3d-8957-4872-a015-c3fe53badeca None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.569723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 422/843] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:14.575179 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7204ab-2f57-4f3c-9ba2-39d34303cae8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.670458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:14.679438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7204ab-2f57-4f3c-9ba2-39d34303cae8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.714982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07679ff-136f-411d-9a02-30641ce2a521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.715811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07679ff-136f-411d-9a02-30641ce2a521 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.716439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07679ff-136f-411d-9a02-30641ce2a521 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.716717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07679ff-136f-411d-9a02-30641ce2a521 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.724460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07679ff-136f-411d-9a02-30641ce2a521 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.725886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07679ff-136f-411d-9a02-30641ce2a521 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c07679ff-136f-411d-9a02-30641ce2a521 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:14.740607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:14.741587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:14.742562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:14.742562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:14.742562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:14.742562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:14.742562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:14.742562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:14.742562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:14.742562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:14.742562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:14.742562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:14.742562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 422/844] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:14.748286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03436b3d-8957-4872-a015-c3fe53badeca None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.850585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03436b3d-8957-4872-a015-c3fe53badeca None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.887605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955ce70-f4b2-4cf7-81d5-850ea5273453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.888444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955ce70-f4b2-4cf7-81d5-850ea5273453 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.888730 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955ce70-f4b2-4cf7-81d5-850ea5273453 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.889009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955ce70-f4b2-4cf7-81d5-850ea5273453 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.889843 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955ce70-f4b2-4cf7-81d5-850ea5273453 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:14.890152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955ce70-f4b2-4cf7-81d5-850ea5273453 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.891576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955ce70-f4b2-4cf7-81d5-850ea5273453 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.910208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 423/845] 10.222.0.65 () {56 vars in 1135 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:14.915701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07679ff-136f-411d-9a02-30641ce2a521 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.022550 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07679ff-136f-411d-9a02-30641ce2a521 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.057536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726a77a-81c6-48a2-834f-3c6984a44bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.058190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726a77a-81c6-48a2-834f-3c6984a44bdb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.058470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726a77a-81c6-48a2-834f-3c6984a44bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.058745 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726a77a-81c6-48a2-834f-3c6984a44bdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.081538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:15.130642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726a77a-81c6-48a2-834f-3c6984a44bdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.132205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726a77a-81c6-48a2-834f-3c6984a44bdb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.230844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 423/846] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:15.237265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955ce70-f4b2-4cf7-81d5-850ea5273453 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.288452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:15.326067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705b69e-6257-4055-b627-a1a5fd224dbf None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.326401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705b69e-6257-4055-b627-a1a5fd224dbf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.326617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705b69e-6257-4055-b627-a1a5fd224dbf None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.328266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 424/847] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:15.336400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01486baa-3800-4d9c-a09b-972dea6e0ccf None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.336681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01486baa-3800-4d9c-a09b-972dea6e0ccf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.336952 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01486baa-3800-4d9c-a09b-972dea6e0ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.354822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955ce70-f4b2-4cf7-81d5-850ea5273453 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.358732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-01486baa-3800-4d9c-a09b-972dea6e0ccf None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:15.360095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-01486baa-3800-4d9c-a09b-972dea6e0ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:15.395312 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ad6c4-73a9-4627-b1e9-a7b1097d8547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.395981 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ad6c4-73a9-4627-b1e9-a7b1097d8547 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.396259 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ad6c4-73a9-4627-b1e9-a7b1097d8547 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.396530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ad6c4-73a9-4627-b1e9-a7b1097d8547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.407073 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ad6c4-73a9-4627-b1e9-a7b1097d8547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.408533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ad6c4-73a9-4627-b1e9-a7b1097d8547 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c59ad6c4-73a9-4627-b1e9-a7b1097d8547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:15.428232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:15.429161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:15.430045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:15.430045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:15.430045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:15.430045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:15.430045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 424/848] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:15.432086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:15.435760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ad6c4-73a9-4627-b1e9-a7b1097d8547 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.459706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 425/849] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:15.480722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01486baa-3800-4d9c-a09b-972dea6e0ccf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.594395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01486baa-3800-4d9c-a09b-972dea6e0ccf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.632675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf86b6-8176-44cc-897b-856430756e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.633497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf86b6-8176-44cc-897b-856430756e31 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.633776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf86b6-8176-44cc-897b-856430756e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.634050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf86b6-8176-44cc-897b-856430756e31 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.634710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf86b6-8176-44cc-897b-856430756e31 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.636133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf86b6-8176-44cc-897b-856430756e31 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1bcf86b6-8176-44cc-897b-856430756e31 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:15.649626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:15.650608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:15.651440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 426/850] 10.222.0.65 () {56 vars in 1132 bytes} [Thu Jun 23 10:50:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:15.658254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf86b6-8176-44cc-897b-856430756e31 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.711259 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ad6c4-73a9-4627-b1e9-a7b1097d8547 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.714023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:15.721329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:15.761847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a5f2a-e982-4ee8-98d2-fdef335fffef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.763284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a5f2a-e982-4ee8-98d2-fdef335fffef None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.764034 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a5f2a-e982-4ee8-98d2-fdef335fffef None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.764544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a5f2a-e982-4ee8-98d2-fdef335fffef None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.765859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a5f2a-e982-4ee8-98d2-fdef335fffef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:15.766357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a5f2a-e982-4ee8-98d2-fdef335fffef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.768422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a5f2a-e982-4ee8-98d2-fdef335fffef None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.781484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf86b6-8176-44cc-897b-856430756e31 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.781816 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 425/851] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:15.787875 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a5f2a-e982-4ee8-98d2-fdef335fffef None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.829911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04647457-4899-4468-b301-658a6c646ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.831167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04647457-4899-4468-b301-658a6c646ede None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.831913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04647457-4899-4468-b301-658a6c646ede None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.832799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04647457-4899-4468-b301-658a6c646ede None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.833995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04647457-4899-4468-b301-658a6c646ede None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:15.834503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04647457-4899-4468-b301-658a6c646ede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.837269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04647457-4899-4468-b301-658a6c646ede None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.844153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 427/852] 10.222.0.65 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:15.850230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04647457-4899-4468-b301-658a6c646ede None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.924549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a5f2a-e982-4ee8-98d2-fdef335fffef None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.965908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621178c-07a2-4fdb-9e58-52b33d3f4976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.967512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621178c-07a2-4fdb-9e58-52b33d3f4976 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.967832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621178c-07a2-4fdb-9e58-52b33d3f4976 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.968257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621178c-07a2-4fdb-9e58-52b33d3f4976 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.976045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621178c-07a2-4fdb-9e58-52b33d3f4976 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.977454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04647457-4899-4468-b301-658a6c646ede None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.977564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621178c-07a2-4fdb-9e58-52b33d3f4976 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.988434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b621178c-07a2-4fdb-9e58-52b33d3f4976 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:15.994530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:15.995630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:15.996689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:15.996689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:15.996689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:15.996689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:15.996689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:15.996689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:15.996689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:15.996689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:16.000409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 426/853] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:16.001515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd4594-4694-4efb-92fa-ca1b4d7bfede None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.001797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd4594-4694-4efb-92fa-ca1b4d7bfede None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.002076 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd4594-4694-4efb-92fa-ca1b4d7bfede None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.003332 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 427/854] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:16.007430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd4594-4694-4efb-92fa-ca1b4d7bfede None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:16.023900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078f073-9001-4474-8ad2-f050e4fc9d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:16.024765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078f073-9001-4474-8ad2-f050e4fc9d8f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.025052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078f073-9001-4474-8ad2-f050e4fc9d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.025328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078f073-9001-4474-8ad2-f050e4fc9d8f None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.026031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078f073-9001-4474-8ad2-f050e4fc9d8f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:16.026339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078f073-9001-4474-8ad2-f050e4fc9d8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:16.027748 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078f073-9001-4474-8ad2-f050e4fc9d8f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:16.039690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 428/855] 10.222.0.65 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:16.039690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc0115-c5f0-4691-8e11-475a07e2aa86 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.039690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc0115-c5f0-4691-8e11-475a07e2aa86 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.039690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc0115-c5f0-4691-8e11-475a07e2aa86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.044706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:16.057992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4bdc0115-c5f0-4691-8e11-475a07e2aa86 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:16.059358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4bdc0115-c5f0-4691-8e11-475a07e2aa86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:16.139940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd4594-4694-4efb-92fa-ca1b4d7bfede None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:16.159748 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 429/856] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:16.185789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc0115-c5f0-4691-8e11-475a07e2aa86 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:16.187920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444379f-a10b-4128-b3a3-5ea8b9d075ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:16.188775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444379f-a10b-4128-b3a3-5ea8b9d075ca None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.189093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444379f-a10b-4128-b3a3-5ea8b9d075ca None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.189518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444379f-a10b-4128-b3a3-5ea8b9d075ca None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.190378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444379f-a10b-4128-b3a3-5ea8b9d075ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:16.190699 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444379f-a10b-4128-b3a3-5ea8b9d075ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:16.192212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444379f-a10b-4128-b3a3-5ea8b9d075ca None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:16.212077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 428/857] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:16.213456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444379f-a10b-4128-b3a3-5ea8b9d075ca None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:16.306617 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:16.313950 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc0115-c5f0-4691-8e11-475a07e2aa86 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:16.317493 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:16.327196 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444379f-a10b-4128-b3a3-5ea8b9d075ca None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:16.354189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96db20-29b0-41f1-8aa3-c4e9b2411c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:16.355102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96db20-29b0-41f1-8aa3-c4e9b2411c4b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.355502 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96db20-29b0-41f1-8aa3-c4e9b2411c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.355809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96db20-29b0-41f1-8aa3-c4e9b2411c4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.360535 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc21190-0d3f-45d3-9d8d-dac88455707e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:16.361194 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc21190-0d3f-45d3-9d8d-dac88455707e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.361475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc21190-0d3f-45d3-9d8d-dac88455707e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.361792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc21190-0d3f-45d3-9d8d-dac88455707e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.362573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc21190-0d3f-45d3-9d8d-dac88455707e None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=8458544c021049c587856343e8001429, user.id=d3d8f8acd0484857b451877baf03e895) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:16.362930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc21190-0d3f-45d3-9d8d-dac88455707e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:16.364391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc21190-0d3f-45d3-9d8d-dac88455707e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:16.366955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96db20-29b0-41f1-8aa3-c4e9b2411c4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:16.370465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 429/858] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:16 2022] GET /identity/v3/role_assignments?user.id=d3d8f8acd0484857b451877baf03e895&scope.project.id=8458544c021049c587856343e8001429&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:16.375036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5ff0d-4c60-4c02-a877-547bd3dda3bd None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4f96db20-29b0-41f1-8aa3-c4e9b2411c4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:16.375492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:16.376470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:16.377321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 430/859] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:16.378247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5ff0d-4c60-4c02-a877-547bd3dda3bd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.378742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5ff0d-4c60-4c02-a877-547bd3dda3bd None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.380246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 430/860] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:16.384753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96db20-29b0-41f1-8aa3-c4e9b2411c4b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:16.386594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06357240-f6cb-44ed-9fa9-2ffb85540a2a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.386884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06357240-f6cb-44ed-9fa9-2ffb85540a2a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.387165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06357240-f6cb-44ed-9fa9-2ffb85540a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.410618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-06357240-f6cb-44ed-9fa9-2ffb85540a2a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:16.411857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-06357240-f6cb-44ed-9fa9-2ffb85540a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:16.511705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 431/861] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:16.520942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96db20-29b0-41f1-8aa3-c4e9b2411c4b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:16.535165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06357240-f6cb-44ed-9fa9-2ffb85540a2a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:16.565515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56f93f-c683-4ab9-89d4-692871a9de29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:16.566194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56f93f-c683-4ab9-89d4-692871a9de29 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.566479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56f93f-c683-4ab9-89d4-692871a9de29 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.566752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56f93f-c683-4ab9-89d4-692871a9de29 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.567504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56f93f-c683-4ab9-89d4-692871a9de29 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:16.567844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56f93f-c683-4ab9-89d4-692871a9de29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:16.569224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56f93f-c683-4ab9-89d4-692871a9de29 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:16.578689 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 431/862] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:16.585489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56f93f-c683-4ab9-89d4-692871a9de29 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:16.616969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:16.645512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:16.671557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06357240-f6cb-44ed-9fa9-2ffb85540a2a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:16.707364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0d45-9680-43c4-a0fa-c9049d2744f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:16.708077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0d45-9680-43c4-a0fa-c9049d2744f6 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.708358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0d45-9680-43c4-a0fa-c9049d2744f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.708631 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0d45-9680-43c4-a0fa-c9049d2744f6 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.709351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c0d45-9680-43c4-a0fa-c9049d2744f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:16.710711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c0d45-9680-43c4-a0fa-c9049d2744f6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:16.711156 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56f93f-c683-4ab9-89d4-692871a9de29 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:16.725766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 432/863] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:16 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:16.758956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4dba7e-6aba-40c8-bd90-1056cfff85e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:16.759814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4dba7e-6aba-40c8-bd90-1056cfff85e0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.760102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4dba7e-6aba-40c8-bd90-1056cfff85e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.760383 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4dba7e-6aba-40c8-bd90-1056cfff85e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.773083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4dba7e-6aba-40c8-bd90-1056cfff85e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:16.774754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4dba7e-6aba-40c8-bd90-1056cfff85e0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2e4dba7e-6aba-40c8-bd90-1056cfff85e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:16.794706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:16.795731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:16.796727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:16.796727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:16.796727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:16.796727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:16.796727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 432/864] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:16.803157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0d45-9680-43c4-a0fa-c9049d2744f6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:16.917375 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0d45-9680-43c4-a0fa-c9049d2744f6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:16.955634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9b5c6-2160-4fb7-b0cc-a698ecfd1b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:16.956251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9b5c6-2160-4fb7-b0cc-a698ecfd1b76 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.956535 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9b5c6-2160-4fb7-b0cc-a698ecfd1b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.956806 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9b5c6-2160-4fb7-b0cc-a698ecfd1b76 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.957529 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9b5c6-2160-4fb7-b0cc-a698ecfd1b76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:16.957833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9b5c6-2160-4fb7-b0cc-a698ecfd1b76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:16.959197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9b5c6-2160-4fb7-b0cc-a698ecfd1b76 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:16.962031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:16.974927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 433/865] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:16.980537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4dba7e-6aba-40c8-bd90-1056cfff85e0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:17.060867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:17.087218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4dba7e-6aba-40c8-bd90-1056cfff85e0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:17.123492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350065fd-b76f-446c-a80f-a33780428e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:17.124344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350065fd-b76f-446c-a80f-a33780428e30 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:17.124631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350065fd-b76f-446c-a80f-a33780428e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:17.124906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350065fd-b76f-446c-a80f-a33780428e30 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:17.133430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350065fd-b76f-446c-a80f-a33780428e30 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:17.134880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350065fd-b76f-446c-a80f-a33780428e30 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-350065fd-b76f-446c-a80f-a33780428e30 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:17.149506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:17.150470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:17.151395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:17.151395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:17.151395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:17.151395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:17.151395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:17.151395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:17.151395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:17.151395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:17.151395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:17.151395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:17.151395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 433/866] 10.222.0.65 () {56 vars in 1117 bytes} [Thu Jun 23 10:50:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:17.158605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9b5c6-2160-4fb7-b0cc-a698ecfd1b76 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:17.271857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9b5c6-2160-4fb7-b0cc-a698ecfd1b76 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:17.306799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece98c02-c6c6-46b4-bce1-9aa7d60793d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:17.307734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece98c02-c6c6-46b4-bce1-9aa7d60793d9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:17.308192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece98c02-c6c6-46b4-bce1-9aa7d60793d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:17.308614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece98c02-c6c6-46b4-bce1-9aa7d60793d9 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:17.309664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece98c02-c6c6-46b4-bce1-9aa7d60793d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:17.310119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece98c02-c6c6-46b4-bce1-9aa7d60793d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:17.311714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece98c02-c6c6-46b4-bce1-9aa7d60793d9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:17.329693 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 434/867] 10.222.0.65 () {56 vars in 1149 bytes} [Thu Jun 23 10:50:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:17.336734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350065fd-b76f-446c-a80f-a33780428e30 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:17.441427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350065fd-b76f-446c-a80f-a33780428e30 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:17.476323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003af84-f23d-4fb4-88b0-0b5c9eab540c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:17.477209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003af84-f23d-4fb4-88b0-0b5c9eab540c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:17.477667 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003af84-f23d-4fb4-88b0-0b5c9eab540c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:17.478090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003af84-f23d-4fb4-88b0-0b5c9eab540c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:17.480994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:17.548199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003af84-f23d-4fb4-88b0-0b5c9eab540c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:17.549939 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003af84-f23d-4fb4-88b0-0b5c9eab540c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:17.661143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 434/868] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:17 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:17.668746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece98c02-c6c6-46b4-bce1-9aa7d60793d9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:17.727040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:17.778663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece98c02-c6c6-46b4-bce1-9aa7d60793d9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:17.814077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d27c1f-d611-4f1c-850d-7ae643174ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:17.815108 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d27c1f-d611-4f1c-850d-7ae643174ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:17.815551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d27c1f-d611-4f1c-850d-7ae643174ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:17.815996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d27c1f-d611-4f1c-850d-7ae643174ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:17.827345 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d27c1f-d611-4f1c-850d-7ae643174ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:17.829039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d27c1f-d611-4f1c-850d-7ae643174ce1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c3d27c1f-d611-4f1c-850d-7ae643174ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:17.847101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:17.848244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:17.849371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:17.849371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:17.849371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:17.849371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:17.851241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 435/869] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:17.856765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003af84-f23d-4fb4-88b0-0b5c9eab540c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:17.892603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:17.959193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003af84-f23d-4fb4-88b0-0b5c9eab540c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.001644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4d270-1d65-4ec6-9c50-5eb5ac2007e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.002660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4d270-1d65-4ec6-9c50-5eb5ac2007e6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.003093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4d270-1d65-4ec6-9c50-5eb5ac2007e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.003540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4d270-1d65-4ec6-9c50-5eb5ac2007e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.004508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4d270-1d65-4ec6-9c50-5eb5ac2007e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:18.004965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4d270-1d65-4ec6-9c50-5eb5ac2007e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.006482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4d270-1d65-4ec6-9c50-5eb5ac2007e6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:18.020005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 435/870] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:18.028772 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d27c1f-d611-4f1c-850d-7ae643174ce1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.141478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d27c1f-d611-4f1c-850d-7ae643174ce1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.148695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:18.154786 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d2df8-6ca7-41eb-9bee-06291e9a43a8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.155376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d2df8-6ca7-41eb-9bee-06291e9a43a8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.155910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d2df8-6ca7-41eb-9bee-06291e9a43a8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.157772 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 436/871] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:18.162831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ea2d2-5a2d-409b-8470-ff9e0012fef8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.163365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ea2d2-5a2d-409b-8470-ff9e0012fef8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.163886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ea2d2-5a2d-409b-8470-ff9e0012fef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.180534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc977a1-69bc-477e-b9c9-39e4ef939e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.181337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc977a1-69bc-477e-b9c9-39e4ef939e08 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.181621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc977a1-69bc-477e-b9c9-39e4ef939e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.181897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc977a1-69bc-477e-b9c9-39e4ef939e08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.182373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4d3ea2d2-5a2d-409b-8470-ff9e0012fef8 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:18.183798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4d3ea2d2-5a2d-409b-8470-ff9e0012fef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:18.190761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc977a1-69bc-477e-b9c9-39e4ef939e08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.192239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc977a1-69bc-477e-b9c9-39e4ef939e08 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6fc977a1-69bc-477e-b9c9-39e4ef939e08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:18.210637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:18.211573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:18.212784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:18.212784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:18.212784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:18.212784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:18.212784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:18.212784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:18.212784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:18.212784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:18.212784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 436/872] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:18.217703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc977a1-69bc-477e-b9c9-39e4ef939e08 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.267892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 437/873] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:18 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:18.281912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ea2d2-5a2d-409b-8470-ff9e0012fef8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.308118 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:18.345072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc977a1-69bc-477e-b9c9-39e4ef939e08 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.386268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ea2d2-5a2d-409b-8470-ff9e0012fef8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.390222 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731c58f-2023-4982-992f-332feb7aa40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.390967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731c58f-2023-4982-992f-332feb7aa40d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.391344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731c58f-2023-4982-992f-332feb7aa40d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.391718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731c58f-2023-4982-992f-332feb7aa40d None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.392585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731c58f-2023-4982-992f-332feb7aa40d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:18.392965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731c58f-2023-4982-992f-332feb7aa40d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.394470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731c58f-2023-4982-992f-332feb7aa40d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:18.410719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 437/874] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:18.417071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731c58f-2023-4982-992f-332feb7aa40d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.437861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba00ba6-b083-48eb-9416-9d866c6eb6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.438809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba00ba6-b083-48eb-9416-9d866c6eb6a9 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.439274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba00ba6-b083-48eb-9416-9d866c6eb6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.439801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba00ba6-b083-48eb-9416-9d866c6eb6a9 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.440792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba00ba6-b083-48eb-9416-9d866c6eb6a9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.442421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba00ba6-b083-48eb-9416-9d866c6eb6a9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:18.454024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:18.467684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 438/875] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:18 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:50:18.546454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731c58f-2023-4982-992f-332feb7aa40d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.560238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4cd1d-c79c-4af1-bae3-4e731f712226 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.560558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4cd1d-c79c-4af1-bae3-4e731f712226 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.560805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4cd1d-c79c-4af1-bae3-4e731f712226 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.562602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 439/876] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:18.569472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f889bf0-bed8-45e5-a44b-5e21186d01bf None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.569897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f889bf0-bed8-45e5-a44b-5e21186d01bf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.570279 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f889bf0-bed8-45e5-a44b-5e21186d01bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.589716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2f889bf0-bed8-45e5-a44b-5e21186d01bf None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:18.591631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2f889bf0-bed8-45e5-a44b-5e21186d01bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:18.591779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a03a9-5771-41ea-8d2c-24b42e2339fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.592318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a03a9-5771-41ea-8d2c-24b42e2339fb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.592596 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a03a9-5771-41ea-8d2c-24b42e2339fb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.592868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a03a9-5771-41ea-8d2c-24b42e2339fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.593711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a03a9-5771-41ea-8d2c-24b42e2339fb None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=96655b920bda45cb872624aa4183cb58) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:18.594021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a03a9-5771-41ea-8d2c-24b42e2339fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.595458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a03a9-5771-41ea-8d2c-24b42e2339fb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:18.602378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 438/877] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:18 2022] GET /identity/v3/role_assignments?user.id=96655b920bda45cb872624aa4183cb58&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:18.675244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 440/878] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:18.698083 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a03a9-5771-41ea-8d2c-24b42e2339fb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.792548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:18.808700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a03a9-5771-41ea-8d2c-24b42e2339fb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.843781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e615e9f-1c94-4cae-aa11-f9eabaab44e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.844593 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e615e9f-1c94-4cae-aa11-f9eabaab44e4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.844874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e615e9f-1c94-4cae-aa11-f9eabaab44e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.845149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e615e9f-1c94-4cae-aa11-f9eabaab44e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.845811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e615e9f-1c94-4cae-aa11-f9eabaab44e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.847169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e615e9f-1c94-4cae-aa11-f9eabaab44e4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0e615e9f-1c94-4cae-aa11-f9eabaab44e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:18.860539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:18.861586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:18.863302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 439/879] 10.222.0.65 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:18.871104 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f889bf0-bed8-45e5-a44b-5e21186d01bf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.886646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:18.972233 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b117b5-8190-4cb0-81d3-3d4b30500588 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.972518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b117b5-8190-4cb0-81d3-3d4b30500588 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.972690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b117b5-8190-4cb0-81d3-3d4b30500588 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.974290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 440/880] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:18.977580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f889bf0-bed8-45e5-a44b-5e21186d01bf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.979087 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb9ca9-92eb-455b-b39e-f593c09d40ff None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.979357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb9ca9-92eb-455b-b39e-f593c09d40ff None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.979633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb9ca9-92eb-455b-b39e-f593c09d40ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.997683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3bdb9ca9-92eb-455b-b39e-f593c09d40ff None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:18.998866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3bdb9ca9-92eb-455b-b39e-f593c09d40ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:19.019836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f81356-436e-4da6-b679-d1c97dcb9478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.020506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f81356-436e-4da6-b679-d1c97dcb9478 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.020786 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f81356-436e-4da6-b679-d1c97dcb9478 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.021058 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f81356-436e-4da6-b679-d1c97dcb9478 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.021768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f81356-436e-4da6-b679-d1c97dcb9478 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:19.022075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f81356-436e-4da6-b679-d1c97dcb9478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.023441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f81356-436e-4da6-b679-d1c97dcb9478 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.028927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 441/881] 10.222.0.65 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:19.034638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f81356-436e-4da6-b679-d1c97dcb9478 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.085816 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 441/882] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:19.114847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb9ca9-92eb-455b-b39e-f593c09d40ff None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.151869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:19.180470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f81356-436e-4da6-b679-d1c97dcb9478 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.207457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:19.229066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb9ca9-92eb-455b-b39e-f593c09d40ff None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.231361 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc273feb-ac08-4584-a7a8-c7eca7aa3929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.233931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc273feb-ac08-4584-a7a8-c7eca7aa3929 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.234424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc273feb-ac08-4584-a7a8-c7eca7aa3929 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.234873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc273feb-ac08-4584-a7a8-c7eca7aa3929 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.235779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc273feb-ac08-4584-a7a8-c7eca7aa3929 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:19.236262 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc273feb-ac08-4584-a7a8-c7eca7aa3929 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.237810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc273feb-ac08-4584-a7a8-c7eca7aa3929 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.244820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 442/883] 10.222.0.65 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:19.274853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60d92c-7244-4aee-9855-a45a4d57297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.275919 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60d92c-7244-4aee-9855-a45a4d57297d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.276285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60d92c-7244-4aee-9855-a45a4d57297d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.276635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60d92c-7244-4aee-9855-a45a4d57297d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.277469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60d92c-7244-4aee-9855-a45a4d57297d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.279565 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60d92c-7244-4aee-9855-a45a4d57297d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9a60d92c-7244-4aee-9855-a45a4d57297d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:19.292925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:19.294642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:19.295471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 442/884] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:19.303254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc273feb-ac08-4584-a7a8-c7eca7aa3929 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.414909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc273feb-ac08-4584-a7a8-c7eca7aa3929 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.450882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c25519-7175-40dc-b1d9-d3e030c2bb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.452026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c25519-7175-40dc-b1d9-d3e030c2bb56 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.452510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c25519-7175-40dc-b1d9-d3e030c2bb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.452979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c25519-7175-40dc-b1d9-d3e030c2bb56 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.453879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c25519-7175-40dc-b1d9-d3e030c2bb56 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:19.454386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c25519-7175-40dc-b1d9-d3e030c2bb56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.455967 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c25519-7175-40dc-b1d9-d3e030c2bb56 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.461329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 443/885] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:19.467642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60d92c-7244-4aee-9855-a45a4d57297d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.571064 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60d92c-7244-4aee-9855-a45a4d57297d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.609461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6d928-e76b-4eb0-b757-8be126268b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.610144 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6d928-e76b-4eb0-b757-8be126268b31 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.610431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6d928-e76b-4eb0-b757-8be126268b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.610711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6d928-e76b-4eb0-b757-8be126268b31 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.611422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6d928-e76b-4eb0-b757-8be126268b31 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:19.611765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6d928-e76b-4eb0-b757-8be126268b31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.613193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6d928-e76b-4eb0-b757-8be126268b31 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.622800 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 443/886] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:19.624482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c25519-7175-40dc-b1d9-d3e030c2bb56 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.633438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:19.731292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c25519-7175-40dc-b1d9-d3e030c2bb56 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.769765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cee81-2049-4321-9f5a-ed4509b08f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.770437 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cee81-2049-4321-9f5a-ed4509b08f11 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.770726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cee81-2049-4321-9f5a-ed4509b08f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.771010 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cee81-2049-4321-9f5a-ed4509b08f11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.771840 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cee81-2049-4321-9f5a-ed4509b08f11 None admin] RBAC: Adding query filter params (interface=public, service_id=16bdfc0d76124fb18d10ef3f2810b4bb, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:19.772159 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cee81-2049-4321-9f5a-ed4509b08f11 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.773533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cee81-2049-4321-9f5a-ed4509b08f11 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.780074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 444/887] 10.222.0.65 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/endpoints?service_id=16bdfc0d76124fb18d10ef3f2810b4bb&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:19.785479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6d928-e76b-4eb0-b757-8be126268b31 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.798663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:19.893021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6d928-e76b-4eb0-b757-8be126268b31 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.927693 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4dc85-caa9-4ac1-a693-d412cb18dc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.928388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4dc85-caa9-4ac1-a693-d412cb18dc7f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.928678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4dc85-caa9-4ac1-a693-d412cb18dc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.928966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4dc85-caa9-4ac1-a693-d412cb18dc7f None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.929718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4dc85-caa9-4ac1-a693-d412cb18dc7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.931540 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4dc85-caa9-4ac1-a693-d412cb18dc7f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.937776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 444/888] 10.222.0.65 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 10:50:20.675954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e2a8c-d177-461b-958a-1d52cf4454b6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.676534 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e2a8c-d177-461b-958a-1d52cf4454b6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.677010 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e2a8c-d177-461b-958a-1d52cf4454b6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.678773 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 445/889] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:20.684367 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af96dd-8bff-4211-a369-5e52c3dc9502 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.685426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af96dd-8bff-4211-a369-5e52c3dc9502 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.685939 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af96dd-8bff-4211-a369-5e52c3dc9502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.702970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-98af96dd-8bff-4211-a369-5e52c3dc9502 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:20.704483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-98af96dd-8bff-4211-a369-5e52c3dc9502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:20.773165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 445/890] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:20.792636 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297187aa-ca05-44e6-99b1-070823db23b6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.793117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297187aa-ca05-44e6-99b1-070823db23b6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.793193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297187aa-ca05-44e6-99b1-070823db23b6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.795125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 446/891] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:20.797703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af96dd-8bff-4211-a369-5e52c3dc9502 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:20.800064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a8105-12ac-4630-bfae-29a5eb7ab4ad None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.800357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a8105-12ac-4630-bfae-29a5eb7ab4ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.800642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a8105-12ac-4630-bfae-29a5eb7ab4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.800960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:20.821452 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ae0a8105-12ac-4630-bfae-29a5eb7ab4ad None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:20.823219 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ae0a8105-12ac-4630-bfae-29a5eb7ab4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:20.904471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:20.905440 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af96dd-8bff-4211-a369-5e52c3dc9502 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:20.911687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 447/892] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:20.940941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a8105-12ac-4630-bfae-29a5eb7ab4ad None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:20.947582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92c9a2-8590-441a-873b-ff97cf1f7b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:20.948692 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92c9a2-8590-441a-873b-ff97cf1f7b3e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.949201 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92c9a2-8590-441a-873b-ff97cf1f7b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.949703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92c9a2-8590-441a-873b-ff97cf1f7b3e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.950647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92c9a2-8590-441a-873b-ff97cf1f7b3e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:20.952268 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92c9a2-8590-441a-873b-ff97cf1f7b3e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1e92c9a2-8590-441a-873b-ff97cf1f7b3e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:20.965102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:20.966660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:20.967513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:20.970459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 446/893] 10.222.0.65 () {56 vars in 1132 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:20.976372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92c9a2-8590-441a-873b-ff97cf1f7b3e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.045687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.052982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a8105-12ac-4630-bfae-29a5eb7ab4ad None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.083726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92c9a2-8590-441a-873b-ff97cf1f7b3e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.091368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31c69d-8fda-4836-b6d9-f91d2dfe4e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.092227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31c69d-8fda-4836-b6d9-f91d2dfe4e7f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.092504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31c69d-8fda-4836-b6d9-f91d2dfe4e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.092777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31c69d-8fda-4836-b6d9-f91d2dfe4e7f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.104945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31c69d-8fda-4836-b6d9-f91d2dfe4e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9e31c69d-8fda-4836-b6d9-f91d2dfe4e7f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:21.114075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:21.115019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:21.115900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 448/894] 10.222.0.65 () {56 vars in 1120 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:21.121384 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31c69d-8fda-4836-b6d9-f91d2dfe4e7f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.130873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ff323-b46c-431e-a5ab-3dff9edf3fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.131546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ff323-b46c-431e-a5ab-3dff9edf3fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.131856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ff323-b46c-431e-a5ab-3dff9edf3fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.132132 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ff323-b46c-431e-a5ab-3dff9edf3fc1 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.132839 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ff323-b46c-431e-a5ab-3dff9edf3fc1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:21.133143 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ff323-b46c-431e-a5ab-3dff9edf3fc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.135043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ff323-b46c-431e-a5ab-3dff9edf3fc1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.146123 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 447/895] 10.222.0.65 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:21.147260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ff323-b46c-431e-a5ab-3dff9edf3fc1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.193388 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.239988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31c69d-8fda-4836-b6d9-f91d2dfe4e7f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.252098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ff323-b46c-431e-a5ab-3dff9edf3fc1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.276423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa4e4d-d90d-4c73-8432-7b9354bcf7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.277223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa4e4d-d90d-4c73-8432-7b9354bcf7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.277473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa4e4d-d90d-4c73-8432-7b9354bcf7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.277729 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa4e4d-d90d-4c73-8432-7b9354bcf7f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.279117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa4e4d-d90d-4c73-8432-7b9354bcf7f8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:21.279389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa4e4d-d90d-4c73-8432-7b9354bcf7f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.280937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa4e4d-d90d-4c73-8432-7b9354bcf7f8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.284275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.290492 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce1e72-b4f0-4928-832a-b1c6e49f13c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.295863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce1e72-b4f0-4928-832a-b1c6e49f13c3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.295983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 449/896] 10.222.0.65 () {56 vars in 1115 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:21.296187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce1e72-b4f0-4928-832a-b1c6e49f13c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.296436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce1e72-b4f0-4928-832a-b1c6e49f13c3 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.297087 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce1e72-b4f0-4928-832a-b1c6e49f13c3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:21.297395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce1e72-b4f0-4928-832a-b1c6e49f13c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.298773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce1e72-b4f0-4928-832a-b1c6e49f13c3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.300850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa4e4d-d90d-4c73-8432-7b9354bcf7f8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.309417 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 448/897] 10.222.0.65 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:21.312886 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce1e72-b4f0-4928-832a-b1c6e49f13c3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.411562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa4e4d-d90d-4c73-8432-7b9354bcf7f8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.423170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce1e72-b4f0-4928-832a-b1c6e49f13c3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.433344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.445968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb7c5f-364a-4b05-ad5d-7f185662bd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.446689 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb7c5f-364a-4b05-ad5d-7f185662bd80 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.446968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb7c5f-364a-4b05-ad5d-7f185662bd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.447243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb7c5f-364a-4b05-ad5d-7f185662bd80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.458298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb7c5f-364a-4b05-ad5d-7f185662bd80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.458458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27784dd0-6e2c-4807-89d1-10dfd4640325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.459013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784dd0-6e2c-4807-89d1-10dfd4640325 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.459300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784dd0-6e2c-4807-89d1-10dfd4640325 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.459581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784dd0-6e2c-4807-89d1-10dfd4640325 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.459761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb7c5f-364a-4b05-ad5d-7f185662bd80 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.460407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27784dd0-6e2c-4807-89d1-10dfd4640325 None admin] RBAC: Adding query filter params (interface=public, service_id=6086a99382d146709c37c7a35d64fdae, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:21.460726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27784dd0-6e2c-4807-89d1-10dfd4640325 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.462069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27784dd0-6e2c-4807-89d1-10dfd4640325 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.472207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 449/898] 10.222.0.65 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/endpoints?service_id=6086a99382d146709c37c7a35d64fdae&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:21.472207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f96e46-b9f6-474b-9595-160d27877011 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.472207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f96e46-b9f6-474b-9595-160d27877011 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.472207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f96e46-b9f6-474b-9595-160d27877011 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.472207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 450/899] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:21.475498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f96e46-b9f6-474b-9595-160d27877011 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c1fb7c5f-364a-4b05-ad5d-7f185662bd80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:21.482810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:21.483998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:21.485583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:21.485583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:21.485583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:21.485583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:21.485583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 450/900] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:21.485873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bec77-823e-426f-9880-659acfff51ce None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.486107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bec77-823e-426f-9880-659acfff51ce None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.486380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bec77-823e-426f-9880-659acfff51ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.503207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b14bec77-823e-426f-9880-659acfff51ce None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:21.504397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b14bec77-823e-426f-9880-659acfff51ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:21.541975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.575701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 451/901] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:21.576867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bec77-823e-426f-9880-659acfff51ce None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.601878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f96e46-b9f6-474b-9595-160d27877011 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.637582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33e408-b615-4136-92b4-76c1ef2a4fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.638308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33e408-b615-4136-92b4-76c1ef2a4fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.638528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33e408-b615-4136-92b4-76c1ef2a4fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.638803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33e408-b615-4136-92b4-76c1ef2a4fd2 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.639485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33e408-b615-4136-92b4-76c1ef2a4fd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.640890 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33e408-b615-4136-92b4-76c1ef2a4fd2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.646780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 451/902] 10.222.0.65 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 10:50:21.647518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33e408-b615-4136-92b4-76c1ef2a4fd2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.728699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bec77-823e-426f-9880-659acfff51ce None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.733979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.770273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6b6c9-b6f7-434a-8118-43ceb823f286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.771174 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6b6c9-b6f7-434a-8118-43ceb823f286 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.771560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6b6c9-b6f7-434a-8118-43ceb823f286 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.771993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6b6c9-b6f7-434a-8118-43ceb823f286 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.772837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6b6c9-b6f7-434a-8118-43ceb823f286 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:21.773251 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6b6c9-b6f7-434a-8118-43ceb823f286 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.774732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6b6c9-b6f7-434a-8118-43ceb823f286 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.777100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33e408-b615-4136-92b4-76c1ef2a4fd2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.793312 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 452/903] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:21.795030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6b6c9-b6f7-434a-8118-43ceb823f286 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.809299 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.827333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158b051-fb86-441e-a6eb-87e482cd765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.828239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158b051-fb86-441e-a6eb-87e482cd765f None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.828623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158b051-fb86-441e-a6eb-87e482cd765f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.828906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158b051-fb86-441e-a6eb-87e482cd765f None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.829643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158b051-fb86-441e-a6eb-87e482cd765f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.831012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158b051-fb86-441e-a6eb-87e482cd765f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.847804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 452/904] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:21 2022] POST /identity/v3/services => generated 241 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:21.924089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6b6c9-b6f7-434a-8118-43ceb823f286 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.969885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc724bff-060b-4293-879f-34f0f0057df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.970717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc724bff-060b-4293-879f-34f0f0057df7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.971001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc724bff-060b-4293-879f-34f0f0057df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.971344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc724bff-060b-4293-879f-34f0f0057df7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.980537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc724bff-060b-4293-879f-34f0f0057df7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.981972 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc724bff-060b-4293-879f-34f0f0057df7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-bc724bff-060b-4293-879f-34f0f0057df7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:21.998579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:21.999585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:22.000570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:22.000570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:22.000570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:22.000570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:22.000570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:22.000570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:22.000570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:22.000570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:22.000570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:22.000570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:22.000570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 453/905] 10.222.0.65 () {56 vars in 1117 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:22.005971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158b051-fb86-441e-a6eb-87e482cd765f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.024500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43b529-2562-460b-9577-4fd13af18973 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.024794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43b529-2562-460b-9577-4fd13af18973 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.025071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43b529-2562-460b-9577-4fd13af18973 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.027234 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 454/906] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:22.033641 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e797560-7873-4c14-a5f1-24f25efff379 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.034003 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e797560-7873-4c14-a5f1-24f25efff379 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.034764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e797560-7873-4c14-a5f1-24f25efff379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.054471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9e797560-7873-4c14-a5f1-24f25efff379 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:22.055824 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9e797560-7873-4c14-a5f1-24f25efff379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:22.073818 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.136966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158b051-fb86-441e-a6eb-87e482cd765f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.141602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 455/907] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:22.167111 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e797560-7873-4c14-a5f1-24f25efff379 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.181859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0a2ed-49e5-4b3c-beab-64f410f52b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.182537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0a2ed-49e5-4b3c-beab-64f410f52b12 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.182817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0a2ed-49e5-4b3c-beab-64f410f52b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.183093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0a2ed-49e5-4b3c-beab-64f410f52b12 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.183962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0a2ed-49e5-4b3c-beab-64f410f52b12 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:22.184273 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0a2ed-49e5-4b3c-beab-64f410f52b12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.185747 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0a2ed-49e5-4b3c-beab-64f410f52b12 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.208102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 453/908] 10.222.0.65 () {56 vars in 1149 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:22.213643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0a2ed-49e5-4b3c-beab-64f410f52b12 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.260562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.277935 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e797560-7873-4c14-a5f1-24f25efff379 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.312533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7626d0b-172f-4804-a6a5-8d7176067141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.313427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7626d0b-172f-4804-a6a5-8d7176067141 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.313716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7626d0b-172f-4804-a6a5-8d7176067141 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.313987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7626d0b-172f-4804-a6a5-8d7176067141 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.314640 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7626d0b-172f-4804-a6a5-8d7176067141 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.316120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7626d0b-172f-4804-a6a5-8d7176067141 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.326871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0a2ed-49e5-4b3c-beab-64f410f52b12 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e7626d0b-172f-4804-a6a5-8d7176067141 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:22.329344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:22.330305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:22.331133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 456/909] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:22.338244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7626d0b-172f-4804-a6a5-8d7176067141 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.357594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.365383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f89eca-73e0-48f0-b4ff-18cc4ab72a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.366046 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f89eca-73e0-48f0-b4ff-18cc4ab72a49 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.366332 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f89eca-73e0-48f0-b4ff-18cc4ab72a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.366611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f89eca-73e0-48f0-b4ff-18cc4ab72a49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.447569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f89eca-73e0-48f0-b4ff-18cc4ab72a49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.449198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f89eca-73e0-48f0-b4ff-18cc4ab72a49 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.462317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7626d0b-172f-4804-a6a5-8d7176067141 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.492363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.502445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecc795-7b66-4b74-88da-a4866b49cd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.503364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecc795-7b66-4b74-88da-a4866b49cd7f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.503877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecc795-7b66-4b74-88da-a4866b49cd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.504426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecc795-7b66-4b74-88da-a4866b49cd7f None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.505324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecc795-7b66-4b74-88da-a4866b49cd7f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:22.505844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecc795-7b66-4b74-88da-a4866b49cd7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.507569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecc795-7b66-4b74-88da-a4866b49cd7f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.518117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 457/910] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:22.519252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecc795-7b66-4b74-88da-a4866b49cd7f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.554501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 454/911] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 10:50:22.560064 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f89eca-73e0-48f0-b4ff-18cc4ab72a49 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.601891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.628029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecc795-7b66-4b74-88da-a4866b49cd7f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.663916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9db28-68df-4137-811f-3bd667603741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.664945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9db28-68df-4137-811f-3bd667603741 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.665449 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9db28-68df-4137-811f-3bd667603741 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.665868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9db28-68df-4137-811f-3bd667603741 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.666712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9db28-68df-4137-811f-3bd667603741 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:22.667172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9db28-68df-4137-811f-3bd667603741 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.669488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9db28-68df-4137-811f-3bd667603741 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.672088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f89eca-73e0-48f0-b4ff-18cc4ab72a49 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.679899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 458/912] 10.222.0.65 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:22.680945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9db28-68df-4137-811f-3bd667603741 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.711577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db5637-4e9b-4f94-b823-4b68dbf001de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.712499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db5637-4e9b-4f94-b823-4b68dbf001de None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.712836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db5637-4e9b-4f94-b823-4b68dbf001de None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.713184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db5637-4e9b-4f94-b823-4b68dbf001de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.722506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.724769 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db5637-4e9b-4f94-b823-4b68dbf001de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.726544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db5637-4e9b-4f94-b823-4b68dbf001de None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e8db5637-4e9b-4f94-b823-4b68dbf001de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:22.746410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:22.748125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:22.749240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:22.749240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:22.749240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:22.749240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:22.754913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 455/913] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:22.755842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db5637-4e9b-4f94-b823-4b68dbf001de None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.793266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9db28-68df-4137-811f-3bd667603741 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.831587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb654a-d06c-42db-81da-d4c5f9c81c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.833126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb654a-d06c-42db-81da-d4c5f9c81c2f None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.833592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb654a-d06c-42db-81da-d4c5f9c81c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.834058 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb654a-d06c-42db-81da-d4c5f9c81c2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.834969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb654a-d06c-42db-81da-d4c5f9c81c2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.836523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb654a-d06c-42db-81da-d4c5f9c81c2f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.851142 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.871979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 459/914] 10.222.0.65 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:22.877204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db5637-4e9b-4f94-b823-4b68dbf001de None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.923949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa20a0-182b-4422-908c-2f9da196a458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.924630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa20a0-182b-4422-908c-2f9da196a458 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.924929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa20a0-182b-4422-908c-2f9da196a458 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.925370 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa20a0-182b-4422-908c-2f9da196a458 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.926006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa20a0-182b-4422-908c-2f9da196a458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:22.926321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa20a0-182b-4422-908c-2f9da196a458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.927804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa20a0-182b-4422-908c-2f9da196a458 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.945975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 456/915] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:22.951401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb654a-d06c-42db-81da-d4c5f9c81c2f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:23.069489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb654a-d06c-42db-81da-d4c5f9c81c2f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:23.107722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518718b5-4e88-4dea-b1d9-88173560899d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:23.108793 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518718b5-4e88-4dea-b1d9-88173560899d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:23.109074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518718b5-4e88-4dea-b1d9-88173560899d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:23.109347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518718b5-4e88-4dea-b1d9-88173560899d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:23.116310 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518718b5-4e88-4dea-b1d9-88173560899d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:23.117745 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518718b5-4e88-4dea-b1d9-88173560899d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-518718b5-4e88-4dea-b1d9-88173560899d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:23.132077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:23.132982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:23.133884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:23.133884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:23.133884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:23.133884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:23.133884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:23.133884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:23.133884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:23.133884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:23.133884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 460/916] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:23.138940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa20a0-182b-4422-908c-2f9da196a458 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:23.191211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:23.244718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa20a0-182b-4422-908c-2f9da196a458 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:23.278155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763a3a8-6689-4840-883c-baf5db53b9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:23.279385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763a3a8-6689-4840-883c-baf5db53b9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:23.279691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763a3a8-6689-4840-883c-baf5db53b9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:23.280020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763a3a8-6689-4840-883c-baf5db53b9c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:23.280808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763a3a8-6689-4840-883c-baf5db53b9c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:23.281116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763a3a8-6689-4840-883c-baf5db53b9c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:23.282539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763a3a8-6689-4840-883c-baf5db53b9c6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:23.296118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 457/917] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:23.304766 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518718b5-4e88-4dea-b1d9-88173560899d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:23.365873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:23.409011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518718b5-4e88-4dea-b1d9-88173560899d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:23.445537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fa06d-3f4f-45ff-8d17-b2cff1484e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:23.446540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fa06d-3f4f-45ff-8d17-b2cff1484e22 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:23.447021 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fa06d-3f4f-45ff-8d17-b2cff1484e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:23.447519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fa06d-3f4f-45ff-8d17-b2cff1484e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:23.448577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fa06d-3f4f-45ff-8d17-b2cff1484e22 None admin] RBAC: Adding query filter params (role.id=7218997909e546eaa53089a27c7886e9, scope.project.id=8458544c021049c587856343e8001429, user.id=96655b920bda45cb872624aa4183cb58) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:23.449085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fa06d-3f4f-45ff-8d17-b2cff1484e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:23.451235 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fa06d-3f4f-45ff-8d17-b2cff1484e22 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:23.457505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 461/918] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:23 2022] GET /identity/v3/role_assignments?user.id=96655b920bda45cb872624aa4183cb58&scope.project.id=8458544c021049c587856343e8001429&role.id=7218997909e546eaa53089a27c7886e9 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:23.752438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2737c-f5a6-415b-9993-67480e217df5 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:23.752780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2737c-f5a6-415b-9993-67480e217df5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:23.752954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2737c-f5a6-415b-9993-67480e217df5 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:23.754577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 458/919] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:23.759829 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a4ec7-6657-4b0b-87cc-c78dc90fcd26 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:23.760126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a4ec7-6657-4b0b-87cc-c78dc90fcd26 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:23.760408 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a4ec7-6657-4b0b-87cc-c78dc90fcd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:23.778015 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-255a4ec7-6657-4b0b-87cc-c78dc90fcd26 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:23.779810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-255a4ec7-6657-4b0b-87cc-c78dc90fcd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:23.848977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 462/920] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:23.873465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2737c-f5a6-415b-9993-67480e217df5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:23.989982 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2737c-f5a6-415b-9993-67480e217df5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:24.000827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:24.017160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4129db-68e5-48cb-906a-7360bedd46ac None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.017456 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4129db-68e5-48cb-906a-7360bedd46ac None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.017724 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4129db-68e5-48cb-906a-7360bedd46ac None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.019228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 463/921] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:24.024344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c488b-3b3b-4b2d-a1bf-c3dd7a9c42f6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.024853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c488b-3b3b-4b2d-a1bf-c3dd7a9c42f6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.025324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c488b-3b3b-4b2d-a1bf-c3dd7a9c42f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.031881 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36835f-7f93-482a-b14e-80133533f520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:24.032726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36835f-7f93-482a-b14e-80133533f520 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.033101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36835f-7f93-482a-b14e-80133533f520 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.033385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36835f-7f93-482a-b14e-80133533f520 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.034051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36835f-7f93-482a-b14e-80133533f520 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:24.035422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36835f-7f93-482a-b14e-80133533f520 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:24.045246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cd7c488b-3b3b-4b2d-a1bf-c3dd7a9c42f6 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:24.046719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cd7c488b-3b3b-4b2d-a1bf-c3dd7a9c42f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1b36835f-7f93-482a-b14e-80133533f520 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:24.057304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:24.058284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:24.059113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 459/922] 10.222.0.65 () {56 vars in 1132 bytes} [Thu Jun 23 10:50:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:24.064995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36835f-7f93-482a-b14e-80133533f520 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:24.112492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:24.136226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 464/923] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:24.165807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c488b-3b3b-4b2d-a1bf-c3dd7a9c42f6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:24.180996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36835f-7f93-482a-b14e-80133533f520 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:24.219778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ac4ec-af33-436f-925e-a7f2dfea1409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:24.220456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ac4ec-af33-436f-925e-a7f2dfea1409 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.220741 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ac4ec-af33-436f-925e-a7f2dfea1409 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.221022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ac4ec-af33-436f-925e-a7f2dfea1409 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.221729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ac4ec-af33-436f-925e-a7f2dfea1409 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:24.222039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ac4ec-af33-436f-925e-a7f2dfea1409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:24.223409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ac4ec-af33-436f-925e-a7f2dfea1409 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:24.228778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 460/924] 10.222.0.65 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:24.234659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ac4ec-af33-436f-925e-a7f2dfea1409 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:24.274368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:24.294238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c488b-3b3b-4b2d-a1bf-c3dd7a9c42f6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:24.341797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c8923-5dfb-4ad0-96c2-fbecf7c8d622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:24.342960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c8923-5dfb-4ad0-96c2-fbecf7c8d622 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.343471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c8923-5dfb-4ad0-96c2-fbecf7c8d622 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.343980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c8923-5dfb-4ad0-96c2-fbecf7c8d622 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.344838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c8923-5dfb-4ad0-96c2-fbecf7c8d622 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:24.346410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c8923-5dfb-4ad0-96c2-fbecf7c8d622 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:24.348973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ac4ec-af33-436f-925e-a7f2dfea1409 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d35c8923-5dfb-4ad0-96c2-fbecf7c8d622 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:24.359616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:24.360638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:24.361505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:24.364368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 465/925] 10.222.0.65 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:24.369831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c8923-5dfb-4ad0-96c2-fbecf7c8d622 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:24.393541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dc400-b404-4049-8f5b-8393474737c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:24.394618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dc400-b404-4049-8f5b-8393474737c6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.394908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dc400-b404-4049-8f5b-8393474737c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.395221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dc400-b404-4049-8f5b-8393474737c6 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.396191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381dc400-b404-4049-8f5b-8393474737c6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:24.396506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381dc400-b404-4049-8f5b-8393474737c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:24.398350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381dc400-b404-4049-8f5b-8393474737c6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:24.409347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 461/926] 10.222.0.65 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:24.411520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dc400-b404-4049-8f5b-8393474737c6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:24.425995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:24.487445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c8923-5dfb-4ad0-96c2-fbecf7c8d622 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:24.525267 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2066d4e-0326-46c6-90cb-53210d16cf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:24.526381 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2066d4e-0326-46c6-90cb-53210d16cf90 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.526917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2066d4e-0326-46c6-90cb-53210d16cf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.527396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2066d4e-0326-46c6-90cb-53210d16cf90 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.528456 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2066d4e-0326-46c6-90cb-53210d16cf90 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:24.529016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2066d4e-0326-46c6-90cb-53210d16cf90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:24.530612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2066d4e-0326-46c6-90cb-53210d16cf90 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:24.531852 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:24.534984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dc400-b404-4049-8f5b-8393474737c6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:24.543266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 466/927] 10.222.0.65 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:24.544807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2066d4e-0326-46c6-90cb-53210d16cf90 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:24.582177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb1038-c697-4c56-ae35-9d7cd3d8a30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:24.582968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb1038-c697-4c56-ae35-9d7cd3d8a30e None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.583745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb1038-c697-4c56-ae35-9d7cd3d8a30e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.584115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb1038-c697-4c56-ae35-9d7cd3d8a30e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.585283 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb1038-c697-4c56-ae35-9d7cd3d8a30e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:24.586790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb1038-c697-4c56-ae35-9d7cd3d8a30e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:24.619748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 462/928] 10.222.0.65 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:24.671691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2066d4e-0326-46c6-90cb-53210d16cf90 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:24.685031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:24.718673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad593f-9218-4dd0-91e5-ff52a71c0fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:24.719365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad593f-9218-4dd0-91e5-ff52a71c0fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.719681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad593f-9218-4dd0-91e5-ff52a71c0fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.719966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad593f-9218-4dd0-91e5-ff52a71c0fe9 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.720690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad593f-9218-4dd0-91e5-ff52a71c0fe9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:24.721006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad593f-9218-4dd0-91e5-ff52a71c0fe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:24.722484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad593f-9218-4dd0-91e5-ff52a71c0fe9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:24.728597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 467/929] 10.222.0.65 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:24.736831 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb1038-c697-4c56-ae35-9d7cd3d8a30e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:24.845921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb1038-c697-4c56-ae35-9d7cd3d8a30e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:24.883273 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89877577-7f3f-4519-93f3-f7d695b9c8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:24.884253 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89877577-7f3f-4519-93f3-f7d695b9c8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.884709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89877577-7f3f-4519-93f3-f7d695b9c8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.885135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89877577-7f3f-4519-93f3-f7d695b9c8c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.886066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89877577-7f3f-4519-93f3-f7d695b9c8c8 None admin] RBAC: Adding query filter params (interface=public, service_id=92ce67868bf94924bb4bac6e63df135d, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:24.886579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89877577-7f3f-4519-93f3-f7d695b9c8c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:24.888214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89877577-7f3f-4519-93f3-f7d695b9c8c8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:24.898261 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 463/930] 10.222.0.65 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/endpoints?service_id=92ce67868bf94924bb4bac6e63df135d&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:24.899782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad593f-9218-4dd0-91e5-ff52a71c0fe9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.005652 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad593f-9218-4dd0-91e5-ff52a71c0fe9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.040859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99445bfb-a323-4090-8669-3e7193a85285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.041778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99445bfb-a323-4090-8669-3e7193a85285 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.042233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99445bfb-a323-4090-8669-3e7193a85285 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.042656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99445bfb-a323-4090-8669-3e7193a85285 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.043484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99445bfb-a323-4090-8669-3e7193a85285 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:25.045678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99445bfb-a323-4090-8669-3e7193a85285 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:25.056494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 468/931] 10.222.0.65 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 10:50:25.076646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf0463-6f77-4bb6-86e9-e23302d93979 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.077660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf0463-6f77-4bb6-86e9-e23302d93979 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.078147 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf0463-6f77-4bb6-86e9-e23302d93979 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.080241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 464/932] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:25.085457 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6612428-06ae-4021-ae19-439dc408ffef None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.085750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6612428-06ae-4021-ae19-439dc408ffef None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.086033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6612428-06ae-4021-ae19-439dc408ffef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.111399 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b6612428-06ae-4021-ae19-439dc408ffef None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:25.112690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b6612428-06ae-4021-ae19-439dc408ffef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:25.144984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:25.191846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 469/933] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:25.214805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf0463-6f77-4bb6-86e9-e23302d93979 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.244005 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:25.330861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf0463-6f77-4bb6-86e9-e23302d93979 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.366656 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6487342-a0b0-4a0a-814d-0b6cb315df41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.367249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6487342-a0b0-4a0a-814d-0b6cb315df41 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.367536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6487342-a0b0-4a0a-814d-0b6cb315df41 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.367846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6487342-a0b0-4a0a-814d-0b6cb315df41 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.368559 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6487342-a0b0-4a0a-814d-0b6cb315df41 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:25.370003 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6487342-a0b0-4a0a-814d-0b6cb315df41 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:25.371376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: WARNING keystone.server.flask.common [None req-e6487342-a0b0-4a0a-814d-0b6cb315df41 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 10:50:25.391200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.sql.core [None req-e6487342-a0b0-4a0a-814d-0b6cb315df41 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 10:50:25.391200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 10:50:25.391200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [parameters: {'id': '9d507d27d06049c080e2f5675f6a51c5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 10:50:25.391200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80206) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e6487342-a0b0-4a0a-814d-0b6cb315df41 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9d507d27d06049c080e2f5675f6a51c5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9d507d27d06049c080e2f5675f6a51c5. Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 10:50:25.399674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 23 10:50:25.400486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application yield resource Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise self.value Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application session.commit() Jun 23 10:50:25.401420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.flush() Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application compat.raise_( Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 10:50:25.402293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 10:50:25.404384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 10:50:25.406276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [parameters: {'id': '9d507d27d06049c080e2f5675f6a51c5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:25.412464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:25.414275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:25.416443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:25.416443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 23 10:50:25.416443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 23 10:50:25.416443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:25.416443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:25.416443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 23 10:50:25.416443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 23 10:50:25.416443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9d507d27d06049c080e2f5675f6a51c5. Jun 23 10:50:25.416443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:25.416443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 465/934] 10.222.0.65 () {60 vars in 1140 bytes} [Thu Jun 23 10:50:25 2022] POST /identity/v3/projects => generated 262 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:25.423712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6612428-06ae-4021-ae19-439dc408ffef None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.526879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6612428-06ae-4021-ae19-439dc408ffef None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.542536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41551b-846d-4ac1-8351-dcb6d6559629 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.542810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41551b-846d-4ac1-8351-dcb6d6559629 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.543074 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41551b-846d-4ac1-8351-dcb6d6559629 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.551925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 466/935] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:25.553118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d6e0d-8b3c-44c3-bcb7-5d2751a76906 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.553400 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d6e0d-8b3c-44c3-bcb7-5d2751a76906 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.553671 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d6e0d-8b3c-44c3-bcb7-5d2751a76906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.564290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a9cfb-7499-4f41-b9cd-8fb5e8eae728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.565254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a9cfb-7499-4f41-b9cd-8fb5e8eae728 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.565886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a9cfb-7499-4f41-b9cd-8fb5e8eae728 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.566170 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a9cfb-7499-4f41-b9cd-8fb5e8eae728 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.570977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c52d6e0d-8b3c-44c3-bcb7-5d2751a76906 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:25.572629 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c52d6e0d-8b3c-44c3-bcb7-5d2751a76906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:25.573454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a9cfb-7499-4f41-b9cd-8fb5e8eae728 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:25.575098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a9cfb-7499-4f41-b9cd-8fb5e8eae728 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ff8a9cfb-7499-4f41-b9cd-8fb5e8eae728 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:25.593473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:25.594560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:25.595669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:25.595669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:25.595669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:25.595669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:25.595669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:25.595669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:25.595669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:25.595669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:25.595669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 470/936] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:25.603110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a9cfb-7499-4f41-b9cd-8fb5e8eae728 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.636147 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:25.649132 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 467/937] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:25.672067 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d6e0d-8b3c-44c3-bcb7-5d2751a76906 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.726429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a9cfb-7499-4f41-b9cd-8fb5e8eae728 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.732020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:25.770526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d6e0d-8b3c-44c3-bcb7-5d2751a76906 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.771301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cc212-5e52-458e-ae3d-30f0fe83b83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.772385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cc212-5e52-458e-ae3d-30f0fe83b83f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.772686 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cc212-5e52-458e-ae3d-30f0fe83b83f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.772973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cc212-5e52-458e-ae3d-30f0fe83b83f None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.775470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cc212-5e52-458e-ae3d-30f0fe83b83f None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:25.775818 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cc212-5e52-458e-ae3d-30f0fe83b83f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:25.777267 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cc212-5e52-458e-ae3d-30f0fe83b83f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:25.802573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 471/938] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:25.816070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258efc7-5e3d-426b-a02f-72837d71822b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.817118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258efc7-5e3d-426b-a02f-72837d71822b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.817464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258efc7-5e3d-426b-a02f-72837d71822b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.817804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258efc7-5e3d-426b-a02f-72837d71822b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.830867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0258efc7-5e3d-426b-a02f-72837d71822b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0258efc7-5e3d-426b-a02f-72837d71822b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:25.839382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:25.840599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:25.842262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:25.842262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:25.842262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:25.842262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:25.842262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:25.842262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:25.842262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:25.842262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:25.842262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:25.842262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:25.842262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:25.842262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:25.843691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 468/939] 10.222.0.65 () {56 vars in 1120 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:25.849679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cc212-5e52-458e-ae3d-30f0fe83b83f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.964448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:25.969771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cc212-5e52-458e-ae3d-30f0fe83b83f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.006428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf85b1-b4d7-4f38-bfc4-8baeb1960e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.007412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf85b1-b4d7-4f38-bfc4-8baeb1960e7b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.007945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf85b1-b4d7-4f38-bfc4-8baeb1960e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.008418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf85b1-b4d7-4f38-bfc4-8baeb1960e7b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.009373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf85b1-b4d7-4f38-bfc4-8baeb1960e7b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:26.009874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf85b1-b4d7-4f38-bfc4-8baeb1960e7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.011416 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf85b1-b4d7-4f38-bfc4-8baeb1960e7b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.020444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 472/940] 10.222.0.65 () {56 vars in 1115 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:26.025626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258efc7-5e3d-426b-a02f-72837d71822b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.095265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.131873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258efc7-5e3d-426b-a02f-72837d71822b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.165281 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad111e2-854d-4ad4-afdc-63d9fccc8029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.166077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad111e2-854d-4ad4-afdc-63d9fccc8029 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.166354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad111e2-854d-4ad4-afdc-63d9fccc8029 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.166626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad111e2-854d-4ad4-afdc-63d9fccc8029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.177396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad111e2-854d-4ad4-afdc-63d9fccc8029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.178961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad111e2-854d-4ad4-afdc-63d9fccc8029 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-aad111e2-854d-4ad4-afdc-63d9fccc8029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:26.196162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:26.197251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:26.198374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:26.198374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:26.198374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:26.198374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:26.198374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 469/941] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:26.203790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf85b1-b4d7-4f38-bfc4-8baeb1960e7b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.306310 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf85b1-b4d7-4f38-bfc4-8baeb1960e7b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.345933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d73aa9-c50c-4036-83eb-81e7f5c01581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.345933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d73aa9-c50c-4036-83eb-81e7f5c01581 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.345933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d73aa9-c50c-4036-83eb-81e7f5c01581 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.345933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d73aa9-c50c-4036-83eb-81e7f5c01581 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.346409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d73aa9-c50c-4036-83eb-81e7f5c01581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:26.346409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d73aa9-c50c-4036-83eb-81e7f5c01581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.346409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d73aa9-c50c-4036-83eb-81e7f5c01581 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.358519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 473/942] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:26.363942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad111e2-854d-4ad4-afdc-63d9fccc8029 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.465118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad111e2-854d-4ad4-afdc-63d9fccc8029 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.492186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.500628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6e6a2-c446-435e-8770-9c311ee737e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.501648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6e6a2-c446-435e-8770-9c311ee737e8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.502169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6e6a2-c446-435e-8770-9c311ee737e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.502601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6e6a2-c446-435e-8770-9c311ee737e8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.510423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6e6a2-c446-435e-8770-9c311ee737e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.512051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6e6a2-c446-435e-8770-9c311ee737e8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-efb6e6a2-c446-435e-8770-9c311ee737e8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:26.526482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:26.527616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:26.528832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:26.528832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:26.528832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:26.528832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:26.528832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:26.528832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:26.528832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:26.528832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:26.528832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:26.528832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:26.531039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 470/943] 10.222.0.65 () {56 vars in 1117 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:26.539695 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d73aa9-c50c-4036-83eb-81e7f5c01581 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.552551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.640847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d73aa9-c50c-4036-83eb-81e7f5c01581 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.675105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb909978-bc11-4966-bb61-58d1090cd935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.675810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb909978-bc11-4966-bb61-58d1090cd935 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.676094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb909978-bc11-4966-bb61-58d1090cd935 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.676369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb909978-bc11-4966-bb61-58d1090cd935 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.677200 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb909978-bc11-4966-bb61-58d1090cd935 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:26.677510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb909978-bc11-4966-bb61-58d1090cd935 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.678934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb909978-bc11-4966-bb61-58d1090cd935 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.697688 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 474/944] 10.222.0.65 () {56 vars in 1149 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:26.704745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6e6a2-c446-435e-8770-9c311ee737e8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.806140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6e6a2-c446-435e-8770-9c311ee737e8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.843124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507d7dd-2f3a-4722-ab88-481cb76b498e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.844818 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58e313-52a5-4d87-af37-8d94a34ddee1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.845102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58e313-52a5-4d87-af37-8d94a34ddee1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.845373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58e313-52a5-4d87-af37-8d94a34ddee1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.846862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 475/945] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:26.847461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507d7dd-2f3a-4722-ab88-481cb76b498e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.847765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507d7dd-2f3a-4722-ab88-481cb76b498e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.848040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507d7dd-2f3a-4722-ab88-481cb76b498e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.852740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3a340-04e8-4228-9e52-6748bf47bd98 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.853263 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3a340-04e8-4228-9e52-6748bf47bd98 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.853743 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3a340-04e8-4228-9e52-6748bf47bd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.861472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507d7dd-2f3a-4722-ab88-481cb76b498e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.862931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507d7dd-2f3a-4722-ab88-481cb76b498e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.872913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-32d3a340-04e8-4228-9e52-6748bf47bd98 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:26.874399 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-32d3a340-04e8-4228-9e52-6748bf47bd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9507d7dd-2f3a-4722-ab88-481cb76b498e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:26.882776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:26.883910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:26.885206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:26.885206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:26.885206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:26.885206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:26.885206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 471/946] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:26.892593 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507d7dd-2f3a-4722-ab88-481cb76b498e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.930740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.943544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.959729 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 476/947] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:26.975300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3a340-04e8-4228-9e52-6748bf47bd98 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.010727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507d7dd-2f3a-4722-ab88-481cb76b498e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.047190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca82a0-bebb-4d33-be0d-ead24521b57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.048451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca82a0-bebb-4d33-be0d-ead24521b57e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.048934 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca82a0-bebb-4d33-be0d-ead24521b57e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.049431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca82a0-bebb-4d33-be0d-ead24521b57e None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.050348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca82a0-bebb-4d33-be0d-ead24521b57e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:27.050874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca82a0-bebb-4d33-be0d-ead24521b57e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.052567 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca82a0-bebb-4d33-be0d-ead24521b57e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.065950 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 472/948] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:27.074599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca82a0-bebb-4d33-be0d-ead24521b57e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.083173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3a340-04e8-4228-9e52-6748bf47bd98 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.131945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff9385-8185-487b-8536-b5e598779dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.133024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff9385-8185-487b-8536-b5e598779dce None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.133480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff9385-8185-487b-8536-b5e598779dce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.133908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff9385-8185-487b-8536-b5e598779dce None admin] PATH_INFO: `/v3/services/compute` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.134719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff9385-8185-487b-8536-b5e598779dce None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.136274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff9385-8185-487b-8536-b5e598779dce None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-87ff9385-8185-487b-8536-b5e598779dce None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:27.149134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:27.150469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:27.151613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:27.159319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 477/949] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:27.160839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff9385-8185-487b-8536-b5e598779dce None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.178009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:27.189547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca82a0-bebb-4d33-be0d-ead24521b57e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.223628 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:27.228274 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f10fc4-831a-482e-8ecd-91b16336f050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.229811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f10fc4-831a-482e-8ecd-91b16336f050 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.230100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f10fc4-831a-482e-8ecd-91b16336f050 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.230378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f10fc4-831a-482e-8ecd-91b16336f050 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.238395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f10fc4-831a-482e-8ecd-91b16336f050 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.239749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f10fc4-831a-482e-8ecd-91b16336f050 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e6f10fc4-831a-482e-8ecd-91b16336f050 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:27.254348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:27.255455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:27.256817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:27.256817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:27.256817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:27.256817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:27.256817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:27.256817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:27.256817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:27.256817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:27.256817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 473/950] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:27.261097 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f10fc4-831a-482e-8ecd-91b16336f050 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.273071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff9385-8185-487b-8536-b5e598779dce None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.308184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94dc1a-36dd-4cbf-aab9-a104ed6eaaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.308879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94dc1a-36dd-4cbf-aab9-a104ed6eaaef None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.309167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94dc1a-36dd-4cbf-aab9-a104ed6eaaef None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.309471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94dc1a-36dd-4cbf-aab9-a104ed6eaaef None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.310181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94dc1a-36dd-4cbf-aab9-a104ed6eaaef None admin] RBAC: Adding query filter params (name=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:27.310489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94dc1a-36dd-4cbf-aab9-a104ed6eaaef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.311896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94dc1a-36dd-4cbf-aab9-a104ed6eaaef None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.320644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 478/951] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:27.320644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5937e-ba4e-4580-8c41-78a5972ffff8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.320644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5937e-ba4e-4580-8c41-78a5972ffff8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.320644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5937e-ba4e-4580-8c41-78a5972ffff8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.323614 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 479/952] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:27.324400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5937e-ba4e-4580-8c41-78a5972ffff8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.369410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f10fc4-831a-482e-8ecd-91b16336f050 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.406294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f798334-1a79-4371-801d-fca413fc02c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.406890 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f798334-1a79-4371-801d-fca413fc02c0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.407175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f798334-1a79-4371-801d-fca413fc02c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.407451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f798334-1a79-4371-801d-fca413fc02c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.408329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f798334-1a79-4371-801d-fca413fc02c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:27.408577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f798334-1a79-4371-801d-fca413fc02c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.410006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f798334-1a79-4371-801d-fca413fc02c0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.422197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 474/953] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:27.423464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf0f34-4d17-4db2-a4fa-5acb1e549ff5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.423858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf0f34-4d17-4db2-a4fa-5acb1e549ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.424112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf0f34-4d17-4db2-a4fa-5acb1e549ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.437191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5937e-ba4e-4580-8c41-78a5972ffff8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.440680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a7bf0f34-4d17-4db2-a4fa-5acb1e549ff5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:27.441854 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a7bf0f34-4d17-4db2-a4fa-5acb1e549ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:27.445456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:27.462628 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:27.475567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfbe40-ec77-4b1b-8ed9-52ca34d1247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.476266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfbe40-ec77-4b1b-8ed9-52ca34d1247b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.476571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfbe40-ec77-4b1b-8ed9-52ca34d1247b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.476851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfbe40-ec77-4b1b-8ed9-52ca34d1247b None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.477550 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cfbe40-ec77-4b1b-8ed9-52ca34d1247b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:27.477859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cfbe40-ec77-4b1b-8ed9-52ca34d1247b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.479207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cfbe40-ec77-4b1b-8ed9-52ca34d1247b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.491704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 480/954] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:27.491704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfbe40-ec77-4b1b-8ed9-52ca34d1247b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.529248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 475/955] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:27.552571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf0f34-4d17-4db2-a4fa-5acb1e549ff5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.631856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfbe40-ec77-4b1b-8ed9-52ca34d1247b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.680733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf0f34-4d17-4db2-a4fa-5acb1e549ff5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.681659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f440c-3c41-449f-9ea4-0cfc9c9283d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.682559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f440c-3c41-449f-9ea4-0cfc9c9283d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.682844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f440c-3c41-449f-9ea4-0cfc9c9283d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.683120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f440c-3c41-449f-9ea4-0cfc9c9283d9 None admin] PATH_INFO: `/v3/projects/8458544c021049c587856343e8001429/users/96655b920bda45cb872624aa4183cb58/roles/7218997909e546eaa53089a27c7886e9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.686035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f440c-3c41-449f-9ea4-0cfc9c9283d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8458544c021049c587856343e8001429, user_id=96655b920bda45cb872624aa4183cb58, role_id=7218997909e546eaa53089a27c7886e9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.687471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f440c-3c41-449f-9ea4-0cfc9c9283d9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.714535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 481/956] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:27 2022] PUT /identity/v3/projects/8458544c021049c587856343e8001429/users/96655b920bda45cb872624aa4183cb58/roles/7218997909e546eaa53089a27c7886e9 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:27.719294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719bffa-e1ce-41ae-9bb0-8a60a1bf60ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.720170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719bffa-e1ce-41ae-9bb0-8a60a1bf60ef None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.720450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719bffa-e1ce-41ae-9bb0-8a60a1bf60ef None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.720748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719bffa-e1ce-41ae-9bb0-8a60a1bf60ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.721404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719bffa-e1ce-41ae-9bb0-8a60a1bf60ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.722824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719bffa-e1ce-41ae-9bb0-8a60a1bf60ef None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2719bffa-e1ce-41ae-9bb0-8a60a1bf60ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:27.736584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:27.737867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:27.738764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:27.742958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 476/957] 10.222.0.65 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:27.744454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f440c-3c41-449f-9ea4-0cfc9c9283d9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.807229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:27.869820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f440c-3c41-449f-9ea4-0cfc9c9283d9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.907050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eab593-f5d8-4db9-94c6-2a1e54d5f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.907745 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eab593-f5d8-4db9-94c6-2a1e54d5f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.908032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eab593-f5d8-4db9-94c6-2a1e54d5f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.908313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eab593-f5d8-4db9-94c6-2a1e54d5f2c1 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.909022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eab593-f5d8-4db9-94c6-2a1e54d5f2c1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:27.909331 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eab593-f5d8-4db9-94c6-2a1e54d5f2c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.910690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eab593-f5d8-4db9-94c6-2a1e54d5f2c1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.916174 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 482/958] 10.222.0.65 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:27.922192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719bffa-e1ce-41ae-9bb0-8a60a1bf60ef None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.926945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:28.027487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719bffa-e1ce-41ae-9bb0-8a60a1bf60ef None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.063421 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752616b-1268-4a7c-857e-44eed4b5441e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.064488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752616b-1268-4a7c-857e-44eed4b5441e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.065008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752616b-1268-4a7c-857e-44eed4b5441e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.065488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752616b-1268-4a7c-857e-44eed4b5441e None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.066404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752616b-1268-4a7c-857e-44eed4b5441e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:28.067009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752616b-1268-4a7c-857e-44eed4b5441e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.068607 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752616b-1268-4a7c-857e-44eed4b5441e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:28.074569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 477/959] 10.222.0.65 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:28.080342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eab593-f5d8-4db9-94c6-2a1e54d5f2c1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:28.080821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92988551-868c-4733-9a1d-c781685be7cf None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.081066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92988551-868c-4733-9a1d-c781685be7cf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.081346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92988551-868c-4733-9a1d-c781685be7cf None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.082900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 478/960] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:28.087753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9aa10-a73c-4ed8-95e7-4b7952aa243e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.088046 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9aa10-a73c-4ed8-95e7-4b7952aa243e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.088323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9aa10-a73c-4ed8-95e7-4b7952aa243e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.105072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-29b9aa10-a73c-4ed8-95e7-4b7952aa243e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:28.106792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-29b9aa10-a73c-4ed8-95e7-4b7952aa243e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:28.183313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:28.191727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 479/961] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:28.212756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eab593-f5d8-4db9-94c6-2a1e54d5f2c1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.215438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9aa10-a73c-4ed8-95e7-4b7952aa243e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:28.224797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:28.252880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632978f-1ab1-4739-a53d-bcadfa36adcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.253903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632978f-1ab1-4739-a53d-bcadfa36adcd None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.254415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632978f-1ab1-4739-a53d-bcadfa36adcd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.254916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632978f-1ab1-4739-a53d-bcadfa36adcd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.255903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632978f-1ab1-4739-a53d-bcadfa36adcd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.257684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632978f-1ab1-4739-a53d-bcadfa36adcd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:28.291852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 483/962] 10.222.0.65 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:28.336818 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9aa10-a73c-4ed8-95e7-4b7952aa243e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.382412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852f31f-1986-42f7-8bd4-6ca77d18a446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.383091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852f31f-1986-42f7-8bd4-6ca77d18a446 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.383370 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852f31f-1986-42f7-8bd4-6ca77d18a446 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.383644 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852f31f-1986-42f7-8bd4-6ca77d18a446 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.384428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852f31f-1986-42f7-8bd4-6ca77d18a446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.385946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852f31f-1986-42f7-8bd4-6ca77d18a446 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:28.405569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.sql.core [None req-a852f31f-1986-42f7-8bd4-6ca77d18a446 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 10:50:28.405569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 10:50:28.405569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [parameters: {'user_id': '3c6feff3be354447b8c8111898afe647', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 10:50:28.405569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80206) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a852f31f-1986-42f7-8bd4-6ca77d18a446 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jun 23 10:50:28.414060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 10:50:28.415041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application yield resource Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise self.value Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application session.commit() Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 10:50:28.416075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.flush() Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application compat.raise_( Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 10:50:28.417021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 10:50:28.417959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 10:50:28.419454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [parameters: {'user_id': '3c6feff3be354447b8c8111898afe647', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:28.422139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 23 10:50:28.424100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 23 10:50:28.427905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 23 10:50:28.427905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 23 10:50:28.427905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 10:50:28.427905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 10:50:28.427905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 10:50:28.427905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:28.427905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 480/963] 10.222.0.65 () {60 vars in 1131 bytes} [Thu Jun 23 10:50:28 2022] POST /identity/v3/users => generated 160 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:28.439840 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632978f-1ab1-4739-a53d-bcadfa36adcd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:28.548443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632978f-1ab1-4739-a53d-bcadfa36adcd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.582627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058f395-a7af-4954-8c18-907af31cc175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.583421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058f395-a7af-4954-8c18-907af31cc175 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.583721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058f395-a7af-4954-8c18-907af31cc175 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.588087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058f395-a7af-4954-8c18-907af31cc175 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.596746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058f395-a7af-4954-8c18-907af31cc175 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.598177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058f395-a7af-4954-8c18-907af31cc175 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4058f395-a7af-4954-8c18-907af31cc175 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:28.613281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:28.615494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:28.617909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:28.617909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:28.617909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:28.617909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:28.617909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:28.617909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:28.617909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:28.617909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:28.617909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:50:28.617909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:28.617909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 484/964] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:28.623977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852f31f-1986-42f7-8bd4-6ca77d18a446 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:28.681385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:28.732029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852f31f-1986-42f7-8bd4-6ca77d18a446 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.767252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd182227-7127-40b8-9c13-ef218caad2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.768215 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd182227-7127-40b8-9c13-ef218caad2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.768669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd182227-7127-40b8-9c13-ef218caad2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.769099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd182227-7127-40b8-9c13-ef218caad2c3 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.770044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd182227-7127-40b8-9c13-ef218caad2c3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:28.770496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd182227-7127-40b8-9c13-ef218caad2c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.772434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd182227-7127-40b8-9c13-ef218caad2c3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:28.811531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 481/965] 10.222.0.65 () {56 vars in 1101 bytes} [Thu Jun 23 10:50:28 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:29.636140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0fc1f-94b2-49b9-aca7-e665ccd339bf None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:29.636661 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0fc1f-94b2-49b9-aca7-e665ccd339bf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:29.636761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0fc1f-94b2-49b9-aca7-e665ccd339bf None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:29.638134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 485/966] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:29.643351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37243c44-658c-4b84-b85b-698dcbed3112 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:29.643747 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37243c44-658c-4b84-b85b-698dcbed3112 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:29.643946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37243c44-658c-4b84-b85b-698dcbed3112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:29.661908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-37243c44-658c-4b84-b85b-698dcbed3112 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:29.663428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-37243c44-658c-4b84-b85b-698dcbed3112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:29.737118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 482/967] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:29.760494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0fc1f-94b2-49b9-aca7-e665ccd339bf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:29.789032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:29.853554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d02e46-d129-4915-906b-66a616688d40 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:29.853883 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d02e46-d129-4915-906b-66a616688d40 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:29.854119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d02e46-d129-4915-906b-66a616688d40 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:29.855619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 483/968] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:29.861160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da27b99-bb89-4408-b81d-b4678704d840 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:29.861450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da27b99-bb89-4408-b81d-b4678704d840 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:29.861728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da27b99-bb89-4408-b81d-b4678704d840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:29.869867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0fc1f-94b2-49b9-aca7-e665ccd339bf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:29.879498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3da27b99-bb89-4408-b81d-b4678704d840 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:29.880708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3da27b99-bb89-4408-b81d-b4678704d840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:29.905302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706423c4-597f-4bea-9110-7079e1d9e8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:29.905894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706423c4-597f-4bea-9110-7079e1d9e8c4 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:29.906179 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706423c4-597f-4bea-9110-7079e1d9e8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:29.906460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706423c4-597f-4bea-9110-7079e1d9e8c4 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:29.907178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706423c4-597f-4bea-9110-7079e1d9e8c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:29.908571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706423c4-597f-4bea-9110-7079e1d9e8c4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:29.924106 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 486/969] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:29 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:29.925847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:29.966197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 484/970] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:29.990828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706423c4-597f-4bea-9110-7079e1d9e8c4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.093171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.110928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706423c4-597f-4bea-9110-7079e1d9e8c4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.146526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc89cd-a8d0-4aa2-a193-69cceb873c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.147342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc89cd-a8d0-4aa2-a193-69cceb873c4a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.147626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc89cd-a8d0-4aa2-a193-69cceb873c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.147933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc89cd-a8d0-4aa2-a193-69cceb873c4a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.159083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc89cd-a8d0-4aa2-a193-69cceb873c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-24cc89cd-a8d0-4aa2-a193-69cceb873c4a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:30.167144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:30.168120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:30.169009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:30.169009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:30.169009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:30.169009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:30.169009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:30.169009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:30.169009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:30.169009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:30.169009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:30.169009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:30.169009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:30.169009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:30.170205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 487/971] 10.222.0.65 () {56 vars in 1120 bytes} [Thu Jun 23 10:50:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:30.174607 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da27b99-bb89-4408-b81d-b4678704d840 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.276262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da27b99-bb89-4408-b81d-b4678704d840 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.311487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f808f0-0bb2-4e07-81e5-ad3e807a4244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.312189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f808f0-0bb2-4e07-81e5-ad3e807a4244 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.312475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f808f0-0bb2-4e07-81e5-ad3e807a4244 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.312757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f808f0-0bb2-4e07-81e5-ad3e807a4244 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.313506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f808f0-0bb2-4e07-81e5-ad3e807a4244 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:30.313818 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f808f0-0bb2-4e07-81e5-ad3e807a4244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.315195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f808f0-0bb2-4e07-81e5-ad3e807a4244 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.328342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 485/972] 10.222.0.65 () {56 vars in 1115 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:30.329877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc89cd-a8d0-4aa2-a193-69cceb873c4a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.384665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5f242-ff81-4d3b-95dd-2952228a8801 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.385084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5f242-ff81-4d3b-95dd-2952228a8801 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.385433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5f242-ff81-4d3b-95dd-2952228a8801 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.386881 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 486/973] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:30.392611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ece2c3-6147-4d66-abe0-b97352312c1b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.392898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ece2c3-6147-4d66-abe0-b97352312c1b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.393177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ece2c3-6147-4d66-abe0-b97352312c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.412255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-42ece2c3-6147-4d66-abe0-b97352312c1b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:30.414098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-42ece2c3-6147-4d66-abe0-b97352312c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:30.435105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc89cd-a8d0-4aa2-a193-69cceb873c4a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.447472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.470342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24020a69-7064-4d5c-8260-9e858dc9f654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.471127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24020a69-7064-4d5c-8260-9e858dc9f654 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.471409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24020a69-7064-4d5c-8260-9e858dc9f654 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.471702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24020a69-7064-4d5c-8260-9e858dc9f654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.482657 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24020a69-7064-4d5c-8260-9e858dc9f654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.484386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24020a69-7064-4d5c-8260-9e858dc9f654 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.490791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.501542 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 487/974] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-24020a69-7064-4d5c-8260-9e858dc9f654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:30.502665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:30.503560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:30.504500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:30.504500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:30.504500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:30.504500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:30.508473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 488/975] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:30.509466 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ece2c3-6147-4d66-abe0-b97352312c1b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.526845 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24020a69-7064-4d5c-8260-9e858dc9f654 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.616926 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ece2c3-6147-4d66-abe0-b97352312c1b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.630800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24020a69-7064-4d5c-8260-9e858dc9f654 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.657839 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca457cd-60ff-41a6-a40d-522be32c1555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.658786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca457cd-60ff-41a6-a40d-522be32c1555 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.659102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca457cd-60ff-41a6-a40d-522be32c1555 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.659501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca457cd-60ff-41a6-a40d-522be32c1555 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.660258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca457cd-60ff-41a6-a40d-522be32c1555 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:30.660563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca457cd-60ff-41a6-a40d-522be32c1555 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.662056 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca457cd-60ff-41a6-a40d-522be32c1555 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.665282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a48891-25a6-4c46-9ce1-469ea9a37698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.666038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a48891-25a6-4c46-9ce1-469ea9a37698 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.666318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a48891-25a6-4c46-9ce1-469ea9a37698 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.666594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a48891-25a6-4c46-9ce1-469ea9a37698 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.667246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a48891-25a6-4c46-9ce1-469ea9a37698 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.669093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a48891-25a6-4c46-9ce1-469ea9a37698 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.678405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 488/976] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:30.679364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca457cd-60ff-41a6-a40d-522be32c1555 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-75a48891-25a6-4c46-9ce1-469ea9a37698 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:30.681887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:30.683089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:30.684184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 489/977] 10.222.0.65 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:30.689220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a48891-25a6-4c46-9ce1-469ea9a37698 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.713936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.755691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.795845 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a48891-25a6-4c46-9ce1-469ea9a37698 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.798118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca457cd-60ff-41a6-a40d-522be32c1555 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.830516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bc981-ffc6-4816-be4d-ca64565c4453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.831176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bc981-ffc6-4816-be4d-ca64565c4453 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.831459 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bc981-ffc6-4816-be4d-ca64565c4453 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.831761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bc981-ffc6-4816-be4d-ca64565c4453 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.832306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6a4b5-1b89-45c1-8716-d9e34af336ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.832487 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2bc981-ffc6-4816-be4d-ca64565c4453 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:30.832759 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2bc981-ffc6-4816-be4d-ca64565c4453 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.833112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6a4b5-1b89-45c1-8716-d9e34af336ae None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.833395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6a4b5-1b89-45c1-8716-d9e34af336ae None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.833671 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6a4b5-1b89-45c1-8716-d9e34af336ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.834104 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2bc981-ffc6-4816-be4d-ca64565c4453 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.839092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 490/978] 10.222.0.65 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:30.842338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6a4b5-1b89-45c1-8716-d9e34af336ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.843889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6a4b5-1b89-45c1-8716-d9e34af336ae None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.844465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bc981-ffc6-4816-be4d-ca64565c4453 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-77d6a4b5-1b89-45c1-8716-d9e34af336ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:30.859672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:30.860804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:30.861970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:30.861970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:30.861970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:30.861970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:30.861970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:30.861970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:30.861970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:30.861970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:30.861970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:30.861970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:30.864237 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 489/979] 10.222.0.65 () {56 vars in 1117 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:30.870088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6a4b5-1b89-45c1-8716-d9e34af336ae None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.955505 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.955985 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bc981-ffc6-4816-be4d-ca64565c4453 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.976126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6a4b5-1b89-45c1-8716-d9e34af336ae None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.982931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.993111 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab48ced-ad70-4787-afdb-cb734f4fff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.993820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab48ced-ad70-4787-afdb-cb734f4fff7b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.994113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab48ced-ad70-4787-afdb-cb734f4fff7b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.994525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab48ced-ad70-4787-afdb-cb734f4fff7b None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.995103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab48ced-ad70-4787-afdb-cb734f4fff7b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:30.995417 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab48ced-ad70-4787-afdb-cb734f4fff7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.996862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab48ced-ad70-4787-afdb-cb734f4fff7b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.002837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 491/980] 10.222.0.65 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:31.008435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab48ced-ad70-4787-afdb-cb734f4fff7b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.016378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e9ec5-ddf0-4caa-8cb0-5061e7c70933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.017067 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e9ec5-ddf0-4caa-8cb0-5061e7c70933 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.017355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e9ec5-ddf0-4caa-8cb0-5061e7c70933 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.017634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e9ec5-ddf0-4caa-8cb0-5061e7c70933 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.018484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e9ec5-ddf0-4caa-8cb0-5061e7c70933 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:31.018797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e9ec5-ddf0-4caa-8cb0-5061e7c70933 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.020271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e9ec5-ddf0-4caa-8cb0-5061e7c70933 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.043137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 490/981] 10.222.0.65 () {56 vars in 1149 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:31.045496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e9ec5-ddf0-4caa-8cb0-5061e7c70933 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.118297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab48ced-ad70-4787-afdb-cb734f4fff7b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.159230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090435eb-8754-4e06-b9ab-3d5f3adc309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.159908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090435eb-8754-4e06-b9ab-3d5f3adc309a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.160190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090435eb-8754-4e06-b9ab-3d5f3adc309a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.160462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090435eb-8754-4e06-b9ab-3d5f3adc309a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.161252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090435eb-8754-4e06-b9ab-3d5f3adc309a None admin] RBAC: Adding query filter params (interface=admin, service_id=92ce67868bf94924bb4bac6e63df135d, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:31.161594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090435eb-8754-4e06-b9ab-3d5f3adc309a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.162903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090435eb-8754-4e06-b9ab-3d5f3adc309a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.163812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e9ec5-ddf0-4caa-8cb0-5061e7c70933 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.169468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 492/982] 10.222.0.65 () {56 vars in 1249 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/endpoints?service_id=92ce67868bf94924bb4bac6e63df135d&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:31.174609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090435eb-8754-4e06-b9ab-3d5f3adc309a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.199948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58056d-5a00-4ad0-a044-d4c3290bab85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.200598 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58056d-5a00-4ad0-a044-d4c3290bab85 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.200878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58056d-5a00-4ad0-a044-d4c3290bab85 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.201154 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58056d-5a00-4ad0-a044-d4c3290bab85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.212025 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:31.240224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:31.273880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58056d-5a00-4ad0-a044-d4c3290bab85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.275318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58056d-5a00-4ad0-a044-d4c3290bab85 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.295384 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090435eb-8754-4e06-b9ab-3d5f3adc309a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.329939 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cb147-76b8-4974-8a46-a5d52fb8e1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.330611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cb147-76b8-4974-8a46-a5d52fb8e1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.330892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cb147-76b8-4974-8a46-a5d52fb8e1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.331168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cb147-76b8-4974-8a46-a5d52fb8e1b9 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.331873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cb147-76b8-4974-8a46-a5d52fb8e1b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.333265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cb147-76b8-4974-8a46-a5d52fb8e1b9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.339713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 493/983] 10.222.0.65 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 10:50:31.386531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 491/984] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 10:50:31.390452 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cb147-76b8-4974-8a46-a5d52fb8e1b9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.500642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cb147-76b8-4974-8a46-a5d52fb8e1b9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.518422 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:31.537001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7ce23-619f-4874-aba7-85c54972404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.537805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7ce23-619f-4874-aba7-85c54972404e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.538090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7ce23-619f-4874-aba7-85c54972404e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.538366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7ce23-619f-4874-aba7-85c54972404e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.551114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7ce23-619f-4874-aba7-85c54972404e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.552576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7ce23-619f-4874-aba7-85c54972404e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-bdf7ce23-619f-4874-aba7-85c54972404e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:31.569890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:31.570822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:31.571791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:31.571791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:31.571791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:31.571791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:31.571791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 494/985] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:31.576783 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58056d-5a00-4ad0-a044-d4c3290bab85 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.620549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:31.682786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58056d-5a00-4ad0-a044-d4c3290bab85 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.722225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2d5ab-8218-408e-9dcf-2997bc1b2040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.722901 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2d5ab-8218-408e-9dcf-2997bc1b2040 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.723186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2d5ab-8218-408e-9dcf-2997bc1b2040 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.723464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2d5ab-8218-408e-9dcf-2997bc1b2040 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.724221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2d5ab-8218-408e-9dcf-2997bc1b2040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:31.724532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2d5ab-8218-408e-9dcf-2997bc1b2040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.725897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2d5ab-8218-408e-9dcf-2997bc1b2040 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.742137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 492/986] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:31.743429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7ce23-619f-4874-aba7-85c54972404e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.844334 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7ce23-619f-4874-aba7-85c54972404e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.877859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223ea5b-ef56-45fa-9fa8-db9502e9b028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.878660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223ea5b-ef56-45fa-9fa8-db9502e9b028 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.878948 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223ea5b-ef56-45fa-9fa8-db9502e9b028 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.879223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223ea5b-ef56-45fa-9fa8-db9502e9b028 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.886014 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223ea5b-ef56-45fa-9fa8-db9502e9b028 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.888035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223ea5b-ef56-45fa-9fa8-db9502e9b028 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8223ea5b-ef56-45fa-9fa8-db9502e9b028 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:31.901853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:31.902867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:31.904717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:31.904717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:31.904717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:31.904717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:31.904717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:31.904717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:31.904717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:31.904717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:31.905661 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 495/987] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:31.909224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2d5ab-8218-408e-9dcf-2997bc1b2040 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.975861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9518ca-944f-49d3-8dd5-421116df3cfd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.976202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9518ca-944f-49d3-8dd5-421116df3cfd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.976435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9518ca-944f-49d3-8dd5-421116df3cfd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.977804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 496/988] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:31.982618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe3393-ae74-40f6-a00c-6b40ee52a786 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.982908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe3393-ae74-40f6-a00c-6b40ee52a786 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.983443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe3393-ae74-40f6-a00c-6b40ee52a786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.000475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-64fe3393-ae74-40f6-a00c-6b40ee52a786 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:32.001665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-64fe3393-ae74-40f6-a00c-6b40ee52a786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:32.012086 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:32.015867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2d5ab-8218-408e-9dcf-2997bc1b2040 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.028871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:32.052290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6532be-a0a4-4712-a91b-b049c7973022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.052959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6532be-a0a4-4712-a91b-b049c7973022 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.053235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6532be-a0a4-4712-a91b-b049c7973022 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.053511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6532be-a0a4-4712-a91b-b049c7973022 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.054301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6532be-a0a4-4712-a91b-b049c7973022 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:32.054608 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6532be-a0a4-4712-a91b-b049c7973022 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.056085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6532be-a0a4-4712-a91b-b049c7973022 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.069021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 493/989] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:32.075562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6532be-a0a4-4712-a91b-b049c7973022 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.083699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 497/990] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:31 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:32.102153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe3393-ae74-40f6-a00c-6b40ee52a786 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.185310 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6532be-a0a4-4712-a91b-b049c7973022 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.208697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe3393-ae74-40f6-a00c-6b40ee52a786 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.220327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c9504-c4e9-4dd3-9bab-03fc9f3a1815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.220987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c9504-c4e9-4dd3-9bab-03fc9f3a1815 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.221270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c9504-c4e9-4dd3-9bab-03fc9f3a1815 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.221545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c9504-c4e9-4dd3-9bab-03fc9f3a1815 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.222355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c9504-c4e9-4dd3-9bab-03fc9f3a1815 None admin] RBAC: Adding query filter params (role.id=7218997909e546eaa53089a27c7886e9, scope.project.id=8458544c021049c587856343e8001429, user.id=96655b920bda45cb872624aa4183cb58) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:32.222667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c9504-c4e9-4dd3-9bab-03fc9f3a1815 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.224136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c9504-c4e9-4dd3-9bab-03fc9f3a1815 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.230063 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 494/991] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/role_assignments?user.id=96655b920bda45cb872624aa4183cb58&scope.project.id=8458544c021049c587856343e8001429&role.id=7218997909e546eaa53089a27c7886e9 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:32.245440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ee0a2-7c80-4eb1-b155-1aae1005ce94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.246277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ee0a2-7c80-4eb1-b155-1aae1005ce94 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.246560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ee0a2-7c80-4eb1-b155-1aae1005ce94 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.247368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ee0a2-7c80-4eb1-b155-1aae1005ce94 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.248084 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ee0a2-7c80-4eb1-b155-1aae1005ce94 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.249505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ee0a2-7c80-4eb1-b155-1aae1005ce94 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7d9ee0a2-7c80-4eb1-b155-1aae1005ce94 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:32.262469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:32.263586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:32.269861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 498/992] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:32.272408 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c9504-c4e9-4dd3-9bab-03fc9f3a1815 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.302822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:32.388002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c9504-c4e9-4dd3-9bab-03fc9f3a1815 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.423122 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5652c5a-b606-4988-8c73-6795e935086f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.423854 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5652c5a-b606-4988-8c73-6795e935086f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.424144 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5652c5a-b606-4988-8c73-6795e935086f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.424423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5652c5a-b606-4988-8c73-6795e935086f None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.425136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5652c5a-b606-4988-8c73-6795e935086f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:32.425444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5652c5a-b606-4988-8c73-6795e935086f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.426818 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5652c5a-b606-4988-8c73-6795e935086f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.431921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 495/993] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:32.437733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ee0a2-7c80-4eb1-b155-1aae1005ce94 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.465120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:32.545258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ee0a2-7c80-4eb1-b155-1aae1005ce94 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.578549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29c693-cef5-4930-a9d0-896cde4d7583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.579218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c693-cef5-4930-a9d0-896cde4d7583 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.579502 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c693-cef5-4930-a9d0-896cde4d7583 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.579811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c693-cef5-4930-a9d0-896cde4d7583 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.580521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29c693-cef5-4930-a9d0-896cde4d7583 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:32.580829 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29c693-cef5-4930-a9d0-896cde4d7583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.582185 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29c693-cef5-4930-a9d0-896cde4d7583 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.587176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 499/994] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:32.593650 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5652c5a-b606-4988-8c73-6795e935086f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.680736 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:32.693789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5652c5a-b606-4988-8c73-6795e935086f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.729169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73e38e-06f0-4068-b774-d0ecb471a562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.729834 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73e38e-06f0-4068-b774-d0ecb471a562 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.730117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73e38e-06f0-4068-b774-d0ecb471a562 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.730395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73e38e-06f0-4068-b774-d0ecb471a562 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.731229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73e38e-06f0-4068-b774-d0ecb471a562 None admin] RBAC: Adding query filter params (interface=public, service_id=ba3e2de044e24aaf91e354f64803dfb4, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:32.731548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73e38e-06f0-4068-b774-d0ecb471a562 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.732946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73e38e-06f0-4068-b774-d0ecb471a562 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.742830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 496/995] 10.222.0.65 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/endpoints?service_id=ba3e2de044e24aaf91e354f64803dfb4&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:32.743733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29c693-cef5-4930-a9d0-896cde4d7583 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.832933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:32.845868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29c693-cef5-4930-a9d0-896cde4d7583 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.879714 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345391c4-b54c-4d39-9d93-fa5ef960c823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.880382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345391c4-b54c-4d39-9d93-fa5ef960c823 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.880663 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345391c4-b54c-4d39-9d93-fa5ef960c823 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.880938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345391c4-b54c-4d39-9d93-fa5ef960c823 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.881619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345391c4-b54c-4d39-9d93-fa5ef960c823 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.882961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345391c4-b54c-4d39-9d93-fa5ef960c823 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.890835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 500/996] 10.222.0.65 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 10:50:33.413942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473271f-e59a-4f0e-af69-8c8fdadb4f12 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:33.414281 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473271f-e59a-4f0e-af69-8c8fdadb4f12 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:33.414462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473271f-e59a-4f0e-af69-8c8fdadb4f12 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:33.415997 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 497/997] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:33.421036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec9650-cdae-4b4f-8aff-c4ce94761391 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:33.421345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec9650-cdae-4b4f-8aff-c4ce94761391 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:33.421604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec9650-cdae-4b4f-8aff-c4ce94761391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:33.438970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1bec9650-cdae-4b4f-8aff-c4ce94761391 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:33.440164 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1bec9650-cdae-4b4f-8aff-c4ce94761391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:33.507006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 501/998] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:33.529740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473271f-e59a-4f0e-af69-8c8fdadb4f12 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:33.630424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473271f-e59a-4f0e-af69-8c8fdadb4f12 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:33.663746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111b42a-1465-4007-8e40-496c822f88de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:33.664600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111b42a-1465-4007-8e40-496c822f88de None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:33.664830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111b42a-1465-4007-8e40-496c822f88de None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:33.665104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111b42a-1465-4007-8e40-496c822f88de None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:33.665754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111b42a-1465-4007-8e40-496c822f88de None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:33.667119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111b42a-1465-4007-8e40-496c822f88de None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b111b42a-1465-4007-8e40-496c822f88de None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:33.679836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:33.680828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:33.681665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 498/999] 10.222.0.65 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:33.686829 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec9650-cdae-4b4f-8aff-c4ce94761391 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:33.775755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:33.796426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec9650-cdae-4b4f-8aff-c4ce94761391 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:33.829551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b3de0-a20f-4015-b3a6-9a23467050e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:33.830218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b3de0-a20f-4015-b3a6-9a23467050e9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:33.830501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b3de0-a20f-4015-b3a6-9a23467050e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:33.830777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b3de0-a20f-4015-b3a6-9a23467050e9 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:33.831474 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b3de0-a20f-4015-b3a6-9a23467050e9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:33.831812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b3de0-a20f-4015-b3a6-9a23467050e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:33.833169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b3de0-a20f-4015-b3a6-9a23467050e9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:33.843198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 502/1000] 10.222.0.65 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:33.844619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111b42a-1465-4007-8e40-496c822f88de None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:33.863990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:33.946867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111b42a-1465-4007-8e40-496c822f88de None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:33.981190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83540227-605a-4272-9f18-c04ec30ea833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:33.981868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83540227-605a-4272-9f18-c04ec30ea833 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:33.982146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83540227-605a-4272-9f18-c04ec30ea833 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:33.982423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83540227-605a-4272-9f18-c04ec30ea833 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:33.983121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83540227-605a-4272-9f18-c04ec30ea833 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:33.983708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83540227-605a-4272-9f18-c04ec30ea833 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:33.985072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83540227-605a-4272-9f18-c04ec30ea833 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:33.990198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 499/1001] 10.222.0.65 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:33.995884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b3de0-a20f-4015-b3a6-9a23467050e9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:34.108037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b3de0-a20f-4015-b3a6-9a23467050e9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:34.142398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d5488-10c0-4ce7-b27c-b4def98b0889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:34.143056 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d5488-10c0-4ce7-b27c-b4def98b0889 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.143336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d5488-10c0-4ce7-b27c-b4def98b0889 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.143611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d5488-10c0-4ce7-b27c-b4def98b0889 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.144374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d5488-10c0-4ce7-b27c-b4def98b0889 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:34.145721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d5488-10c0-4ce7-b27c-b4def98b0889 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:34.161004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 503/1002] 10.222.0.65 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:34.299523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8908b9-7445-43b1-a02f-7ab1c9420d7a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.299910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8908b9-7445-43b1-a02f-7ab1c9420d7a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.300122 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8908b9-7445-43b1-a02f-7ab1c9420d7a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.301768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 500/1003] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:34.306846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2e1e2-5ed8-48d1-be04-cfc8a1fe8c6b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.307139 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2e1e2-5ed8-48d1-be04-cfc8a1fe8c6b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.307424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2e1e2-5ed8-48d1-be04-cfc8a1fe8c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.320191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:34.327889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c2a2e1e2-5ed8-48d1-be04-cfc8a1fe8c6b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:34.329555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c2a2e1e2-5ed8-48d1-be04-cfc8a1fe8c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:34.405164 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 504/1004] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:34 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:34.427282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8908b9-7445-43b1-a02f-7ab1c9420d7a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:34.513383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:34.534616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8908b9-7445-43b1-a02f-7ab1c9420d7a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:34.568184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca707417-0bca-4d88-a8b7-d1763a8eeeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:34.568899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca707417-0bca-4d88-a8b7-d1763a8eeeeb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.569180 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca707417-0bca-4d88-a8b7-d1763a8eeeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.569460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca707417-0bca-4d88-a8b7-d1763a8eeeeb None admin] PATH_INFO: `/v3/services/image` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.570114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca707417-0bca-4d88-a8b7-d1763a8eeeeb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:34.571477 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca707417-0bca-4d88-a8b7-d1763a8eeeeb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ca707417-0bca-4d88-a8b7-d1763a8eeeeb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:34.583544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:34.584767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:34.585645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:34.587739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 501/1005] 10.222.0.65 () {56 vars in 1105 bytes} [Thu Jun 23 10:50:34 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:34.590920 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2e1e2-5ed8-48d1-be04-cfc8a1fe8c6b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:34.694026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2e1e2-5ed8-48d1-be04-cfc8a1fe8c6b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:34.722779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:34.728899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3d577-6892-4300-8327-829c960c4d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:34.729570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3d577-6892-4300-8327-829c960c4d68 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.729855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3d577-6892-4300-8327-829c960c4d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.730135 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3d577-6892-4300-8327-829c960c4d68 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.730843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3d577-6892-4300-8327-829c960c4d68 None admin] RBAC: Adding query filter params (name=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:34.731690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3d577-6892-4300-8327-829c960c4d68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:34.733052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3d577-6892-4300-8327-829c960c4d68 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:34.742627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 505/1006] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:34.744166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca707417-0bca-4d88-a8b7-d1763a8eeeeb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:34.844402 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca707417-0bca-4d88-a8b7-d1763a8eeeeb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:34.879254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a846cf-0c16-4d1c-bc6c-18b0156275ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:34.880054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a846cf-0c16-4d1c-bc6c-18b0156275ea None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.880285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a846cf-0c16-4d1c-bc6c-18b0156275ea None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.880564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a846cf-0c16-4d1c-bc6c-18b0156275ea None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.881300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a846cf-0c16-4d1c-bc6c-18b0156275ea None admin] RBAC: Adding query filter params (type=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:34.881611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a846cf-0c16-4d1c-bc6c-18b0156275ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:34.883033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a846cf-0c16-4d1c-bc6c-18b0156275ea None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:34.896060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 502/1007] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:34.931340 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af2306-5737-47ad-bd35-618b630dc857 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.931934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af2306-5737-47ad-bd35-618b630dc857 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.931934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af2306-5737-47ad-bd35-618b630dc857 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.933483 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 506/1008] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:34.939571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d1704-e7cc-4f8f-b553-86e1e6d612ee None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.939909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d1704-e7cc-4f8f-b553-86e1e6d612ee None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.940159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d1704-e7cc-4f8f-b553-86e1e6d612ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.957628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d79d1704-e7cc-4f8f-b553-86e1e6d612ee None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:34.958865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d79d1704-e7cc-4f8f-b553-86e1e6d612ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:34.961173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:35.034127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 503/1009] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:34 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:35.057260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af2306-5737-47ad-bd35-618b630dc857 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:35.173122 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af2306-5737-47ad-bd35-618b630dc857 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:35.207576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b248c9-52c1-4308-862f-cbe33e9453ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:35.208395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b248c9-52c1-4308-862f-cbe33e9453ff None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.208678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b248c9-52c1-4308-862f-cbe33e9453ff None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.208951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b248c9-52c1-4308-862f-cbe33e9453ff None admin] PATH_INFO: `/v3/services/compute` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.209602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b248c9-52c1-4308-862f-cbe33e9453ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:35.210949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b248c9-52c1-4308-862f-cbe33e9453ff None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b2b248c9-52c1-4308-862f-cbe33e9453ff None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:35.223193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:35.224242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:35.225103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 507/1010] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:35.230159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d1704-e7cc-4f8f-b553-86e1e6d612ee None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:35.333855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d1704-e7cc-4f8f-b553-86e1e6d612ee None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:35.369496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea560e-d1ba-4722-b206-d22a61b33d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:35.370093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea560e-d1ba-4722-b206-d22a61b33d86 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.370375 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea560e-d1ba-4722-b206-d22a61b33d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.370654 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea560e-d1ba-4722-b206-d22a61b33d86 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.371358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea560e-d1ba-4722-b206-d22a61b33d86 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:35.371685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea560e-d1ba-4722-b206-d22a61b33d86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:35.373077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea560e-d1ba-4722-b206-d22a61b33d86 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:35.382413 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 504/1011] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:35.383991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b248c9-52c1-4308-862f-cbe33e9453ff None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:35.445152 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:35.488125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b248c9-52c1-4308-862f-cbe33e9453ff None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:35.522294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005c35a-2873-4448-8579-2c2685ab7b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:35.522987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005c35a-2873-4448-8579-2c2685ab7b99 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.523269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005c35a-2873-4448-8579-2c2685ab7b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.523540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005c35a-2873-4448-8579-2c2685ab7b99 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.524269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005c35a-2873-4448-8579-2c2685ab7b99 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:35.524572 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005c35a-2873-4448-8579-2c2685ab7b99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:35.525957 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005c35a-2873-4448-8579-2c2685ab7b99 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:35.531505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 508/1012] 10.222.0.65 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:35.536984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea560e-d1ba-4722-b206-d22a61b33d86 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:35.539851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:35.641252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea560e-d1ba-4722-b206-d22a61b33d86 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:35.676054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea7903-a9a8-468d-8816-13bbcb1d8515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:35.676709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea7903-a9a8-468d-8816-13bbcb1d8515 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.676991 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea7903-a9a8-468d-8816-13bbcb1d8515 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.677265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea7903-a9a8-468d-8816-13bbcb1d8515 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.678005 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea7903-a9a8-468d-8816-13bbcb1d8515 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:35.679365 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea7903-a9a8-468d-8816-13bbcb1d8515 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:35.703724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 505/1013] 10.222.0.65 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:35 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:36.205010 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5981c-2fde-4941-8ef1-a3531317039d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.205526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5981c-2fde-4941-8ef1-a3531317039d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.205633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5981c-2fde-4941-8ef1-a3531317039d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.207056 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 509/1014] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:36.211972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7910ae4-d0b0-4761-b308-09b567d47bf9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.212348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7910ae4-d0b0-4761-b308-09b567d47bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.212521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7910ae4-d0b0-4761-b308-09b567d47bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.229190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c7910ae4-d0b0-4761-b308-09b567d47bf9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:36.230362 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c7910ae4-d0b0-4761-b308-09b567d47bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:36.285887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:36.312001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 506/1015] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:36 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:36.331405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5981c-2fde-4941-8ef1-a3531317039d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:36.433831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5981c-2fde-4941-8ef1-a3531317039d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:36.458394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:36.470261 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab08f8-76d8-48ea-9b64-88e0c1a69e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:36.471065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab08f8-76d8-48ea-9b64-88e0c1a69e68 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.471346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab08f8-76d8-48ea-9b64-88e0c1a69e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.471621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab08f8-76d8-48ea-9b64-88e0c1a69e68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.472812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab08f8-76d8-48ea-9b64-88e0c1a69e68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:36.474236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab08f8-76d8-48ea-9b64-88e0c1a69e68 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.480648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 510/1016] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:36.482351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7910ae4-d0b0-4761-b308-09b567d47bf9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:36.584913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7910ae4-d0b0-4761-b308-09b567d47bf9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:36.619621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339a28c-9546-4de1-9034-2c956f485c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:36.620317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339a28c-9546-4de1-9034-2c956f485c85 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.620602 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339a28c-9546-4de1-9034-2c956f485c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.620884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339a28c-9546-4de1-9034-2c956f485c85 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.621749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339a28c-9546-4de1-9034-2c956f485c85 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:36.623197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339a28c-9546-4de1-9034-2c956f485c85 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.655704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 507/1017] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:36 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:50:36.923844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f3014-92ce-4cc7-976b-41a25b507866 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.924133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f3014-92ce-4cc7-976b-41a25b507866 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.924419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f3014-92ce-4cc7-976b-41a25b507866 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.925792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 511/1018] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:36.931083 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca21957-ae56-42af-8884-4253e62dabd5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.931380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca21957-ae56-42af-8884-4253e62dabd5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.931678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca21957-ae56-42af-8884-4253e62dabd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.949215 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fca21957-ae56-42af-8884-4253e62dabd5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:36.950495 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fca21957-ae56-42af-8884-4253e62dabd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:36.980546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:37.022931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 508/1019] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:36 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:37.046611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f3014-92ce-4cc7-976b-41a25b507866 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:37.156851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f3014-92ce-4cc7-976b-41a25b507866 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:37.190867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea1b53-04f1-41a9-831b-2281e1383a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:37.191460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea1b53-04f1-41a9-831b-2281e1383a68 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:37.191769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea1b53-04f1-41a9-831b-2281e1383a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:37.192050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea1b53-04f1-41a9-831b-2281e1383a68 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:37.192783 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea1b53-04f1-41a9-831b-2281e1383a68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:37.194138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea1b53-04f1-41a9-831b-2281e1383a68 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:37.214809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 512/1020] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:37 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:37.804053 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86493080-e4ef-45a8-b1b8-d20d6fbcf712 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:37.804616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86493080-e4ef-45a8-b1b8-d20d6fbcf712 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:37.804616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86493080-e4ef-45a8-b1b8-d20d6fbcf712 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:37.806107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 509/1021] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:37.811585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c5da-708f-42c4-84a9-47ecec22d2d5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:37.811912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c5da-708f-42c4-84a9-47ecec22d2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:37.812189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c5da-708f-42c4-84a9-47ecec22d2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:37.828640 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e9f7c5da-708f-42c4-84a9-47ecec22d2d5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:37.829811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e9f7c5da-708f-42c4-84a9-47ecec22d2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:37.878832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:37.906258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 513/1022] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:37.930102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86493080-e4ef-45a8-b1b8-d20d6fbcf712 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:38.034054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86493080-e4ef-45a8-b1b8-d20d6fbcf712 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:38.068125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1ddac-8782-4354-80e7-946ed8a791f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:38.069228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1ddac-8782-4354-80e7-946ed8a791f1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.069726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1ddac-8782-4354-80e7-946ed8a791f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.070200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1ddac-8782-4354-80e7-946ed8a791f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.080411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1ddac-8782-4354-80e7-946ed8a791f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-04c1ddac-8782-4354-80e7-946ed8a791f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:38.095536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:38.096660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:38.097522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 510/1023] 10.222.0.65 () {56 vars in 1120 bytes} [Thu Jun 23 10:50:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:38.102599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c5da-708f-42c4-84a9-47ecec22d2d5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:38.202023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c5da-708f-42c4-84a9-47ecec22d2d5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:38.236813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf44f0-5f9d-46a0-99b3-000aaab25455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:38.237550 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf44f0-5f9d-46a0-99b3-000aaab25455 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.237754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf44f0-5f9d-46a0-99b3-000aaab25455 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.238027 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf44f0-5f9d-46a0-99b3-000aaab25455 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.238769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf44f0-5f9d-46a0-99b3-000aaab25455 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:38.239076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf44f0-5f9d-46a0-99b3-000aaab25455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:38.240469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf44f0-5f9d-46a0-99b3-000aaab25455 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:38.248600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 514/1024] 10.222.0.65 () {56 vars in 1115 bytes} [Thu Jun 23 10:50:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:38.253974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1ddac-8782-4354-80e7-946ed8a791f1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:38.281847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:38.357329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1ddac-8782-4354-80e7-946ed8a791f1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:38.392355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d82528-d7f8-4a6a-b5ef-68a0ff8f0f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:38.393177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d82528-d7f8-4a6a-b5ef-68a0ff8f0f6b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.393464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d82528-d7f8-4a6a-b5ef-68a0ff8f0f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.393754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d82528-d7f8-4a6a-b5ef-68a0ff8f0f6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.405657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d82528-d7f8-4a6a-b5ef-68a0ff8f0f6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:38.407101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d82528-d7f8-4a6a-b5ef-68a0ff8f0f6b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e8d82528-d7f8-4a6a-b5ef-68a0ff8f0f6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:38.424212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:38.425137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:38.426066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:38.426066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:38.426066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:38.426066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:38.427981 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 511/1025] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:38.430937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf44f0-5f9d-46a0-99b3-000aaab25455 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:38.475030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:38.532869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf44f0-5f9d-46a0-99b3-000aaab25455 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:38.565421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d50bd-2360-4717-9643-946bf9c10e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:38.566080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d50bd-2360-4717-9643-946bf9c10e6d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.566359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d50bd-2360-4717-9643-946bf9c10e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.566633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d50bd-2360-4717-9643-946bf9c10e6d None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.567352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d50bd-2360-4717-9643-946bf9c10e6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:38.567673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d50bd-2360-4717-9643-946bf9c10e6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:38.569022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d50bd-2360-4717-9643-946bf9c10e6d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:38.580791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 515/1026] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:38.586110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d82528-d7f8-4a6a-b5ef-68a0ff8f0f6b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:38.667145 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:38.689830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d82528-d7f8-4a6a-b5ef-68a0ff8f0f6b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:38.707053 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494a7f6-ea49-4159-ac29-1527aaffab3d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.707617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494a7f6-ea49-4159-ac29-1527aaffab3d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.707942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494a7f6-ea49-4159-ac29-1527aaffab3d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.709475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 516/1027] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:38.714176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d5611-44db-49f7-95f7-bd3a6017b2cf None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.714469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d5611-44db-49f7-95f7-bd3a6017b2cf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.714751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d5611-44db-49f7-95f7-bd3a6017b2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.726677 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f5e77-91ac-4074-adbc-9d7cd76ad6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:38.727707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f5e77-91ac-4074-adbc-9d7cd76ad6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.728134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f5e77-91ac-4074-adbc-9d7cd76ad6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.728550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f5e77-91ac-4074-adbc-9d7cd76ad6b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.732720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6d3d5611-44db-49f7-95f7-bd3a6017b2cf None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:38.734152 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6d3d5611-44db-49f7-95f7-bd3a6017b2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:38.737183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f5e77-91ac-4074-adbc-9d7cd76ad6b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:38.738622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f5e77-91ac-4074-adbc-9d7cd76ad6b7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4d2f5e77-91ac-4074-adbc-9d7cd76ad6b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:38.753199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:38.754188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:38.755177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:38.755177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:38.755177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:38.755177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:38.755177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:38.755177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:38.755177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:38.755177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:38.755177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:38.755177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:38.755177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 512/1028] 10.222.0.65 () {56 vars in 1093 bytes} [Thu Jun 23 10:50:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:38.759925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f5e77-91ac-4074-adbc-9d7cd76ad6b7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:38.811923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 517/1029] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:38.833836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d5611-44db-49f7-95f7-bd3a6017b2cf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:38.862888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:38.864579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f5e77-91ac-4074-adbc-9d7cd76ad6b7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:38.903503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149752c-2b10-4522-bf23-b381a68af7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:38.904317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149752c-2b10-4522-bf23-b381a68af7de None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.904605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149752c-2b10-4522-bf23-b381a68af7de None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.904882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149752c-2b10-4522-bf23-b381a68af7de None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.905722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149752c-2b10-4522-bf23-b381a68af7de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:38.906031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149752c-2b10-4522-bf23-b381a68af7de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:38.907558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149752c-2b10-4522-bf23-b381a68af7de None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:38.916692 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:38.927754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 513/1030] 10.222.0.65 () {56 vars in 1133 bytes} [Thu Jun 23 10:50:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:38.933455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149752c-2b10-4522-bf23-b381a68af7de None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:38.943455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d5611-44db-49f7-95f7-bd3a6017b2cf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:38.982542 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674425e-4dc4-4b19-af65-918152dbfbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:38.983595 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674425e-4dc4-4b19-af65-918152dbfbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.983919 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674425e-4dc4-4b19-af65-918152dbfbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.984201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674425e-4dc4-4b19-af65-918152dbfbb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.985454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674425e-4dc4-4b19-af65-918152dbfbb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:38.986892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674425e-4dc4-4b19-af65-918152dbfbb7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:38.988875 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 518/1031] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:38.994178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674425e-4dc4-4b19-af65-918152dbfbb7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.041806 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149752c-2b10-4522-bf23-b381a68af7de None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.077735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348dd7c-42d3-417a-a32f-40c31d7333a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.078491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348dd7c-42d3-417a-a32f-40c31d7333a1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.078773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348dd7c-42d3-417a-a32f-40c31d7333a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.079047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348dd7c-42d3-417a-a32f-40c31d7333a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.097816 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:39.102187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674425e-4dc4-4b19-af65-918152dbfbb7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.138167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619647d2-2735-4c45-a757-101d28d660c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.139113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619647d2-2735-4c45-a757-101d28d660c2 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.139403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619647d2-2735-4c45-a757-101d28d660c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.139709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619647d2-2735-4c45-a757-101d28d660c2 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.140426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619647d2-2735-4c45-a757-101d28d660c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.142252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619647d2-2735-4c45-a757-101d28d660c2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.151597 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:39.160471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348dd7c-42d3-417a-a32f-40c31d7333a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.161973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348dd7c-42d3-417a-a32f-40c31d7333a1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.166148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 519/1032] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:50:38 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:50:39.279610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 514/1033] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:38 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:39.285358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619647d2-2735-4c45-a757-101d28d660c2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.323345 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924a2a0-a3f4-4638-8725-64a4dbcc5f99 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.323678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924a2a0-a3f4-4638-8725-64a4dbcc5f99 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.323965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924a2a0-a3f4-4638-8725-64a4dbcc5f99 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.325643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 515/1034] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:39.330536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09785673-ec1c-49b0-b901-a482c738e1b7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.330824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09785673-ec1c-49b0-b901-a482c738e1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.331107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09785673-ec1c-49b0-b901-a482c738e1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.347884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-09785673-ec1c-49b0-b901-a482c738e1b7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:39.349080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-09785673-ec1c-49b0-b901-a482c738e1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:39.406204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619647d2-2735-4c45-a757-101d28d660c2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.422099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 516/1035] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:39.444984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09785673-ec1c-49b0-b901-a482c738e1b7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.445172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83f8a9-a061-4df0-bde4-ad12644396ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.445796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83f8a9-a061-4df0-bde4-ad12644396ea None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.446072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83f8a9-a061-4df0-bde4-ad12644396ea None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.446344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83f8a9-a061-4df0-bde4-ad12644396ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.461028 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83f8a9-a061-4df0-bde4-ad12644396ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.463182 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83f8a9-a061-4df0-bde4-ad12644396ea None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.472477 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-dc83f8a9-a061-4df0-bde4-ad12644396ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:39.482709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:39.484121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:39.485072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:39.485072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:39.485072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:39.485072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:39.485072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 520/1036] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:39.489909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83f8a9-a061-4df0-bde4-ad12644396ea None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.502481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:39.561171 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09785673-ec1c-49b0-b901-a482c738e1b7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.598047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b5263-233c-4cb3-99c4-3afde6a4bd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.598836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b5263-233c-4cb3-99c4-3afde6a4bd6c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.599119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b5263-233c-4cb3-99c4-3afde6a4bd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.599393 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b5263-233c-4cb3-99c4-3afde6a4bd6c None admin] PATH_INFO: `/v3/services/image` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.600082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b5263-233c-4cb3-99c4-3afde6a4bd6c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.601520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b5263-233c-4cb3-99c4-3afde6a4bd6c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.603445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83f8a9-a061-4df0-bde4-ad12644396ea None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4f4b5263-233c-4cb3-99c4-3afde6a4bd6c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:39.613824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:39.614823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:39.615700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 517/1037] 10.222.0.65 () {56 vars in 1105 bytes} [Thu Jun 23 10:50:39 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:39.621275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b5263-233c-4cb3-99c4-3afde6a4bd6c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.642482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a06d2-702c-42c7-957c-bd74f4ff489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.643149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a06d2-702c-42c7-957c-bd74f4ff489b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.643433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a06d2-702c-42c7-957c-bd74f4ff489b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.643742 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a06d2-702c-42c7-957c-bd74f4ff489b None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.644467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a06d2-702c-42c7-957c-bd74f4ff489b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:39.644980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a06d2-702c-42c7-957c-bd74f4ff489b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.646132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a06d2-702c-42c7-957c-bd74f4ff489b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.663238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 521/1038] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:39.664276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a06d2-702c-42c7-957c-bd74f4ff489b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.723271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:39.735410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b5263-233c-4cb3-99c4-3afde6a4bd6c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.736667 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:39.772908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84d157-a948-4341-ae79-33c026638ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.773584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84d157-a948-4341-ae79-33c026638ddd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.773865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84d157-a948-4341-ae79-33c026638ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.774141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84d157-a948-4341-ae79-33c026638ddd None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.774852 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b84d157-a948-4341-ae79-33c026638ddd None admin] RBAC: Adding query filter params (name=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:39.775160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b84d157-a948-4341-ae79-33c026638ddd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.776832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a06d2-702c-42c7-957c-bd74f4ff489b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.776948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b84d157-a948-4341-ae79-33c026638ddd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.782625 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 518/1039] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:39.788175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84d157-a948-4341-ae79-33c026638ddd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.817830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada25c8a-7bb5-40de-9f28-8681bf7ffb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.818545 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada25c8a-7bb5-40de-9f28-8681bf7ffb3f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.818827 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada25c8a-7bb5-40de-9f28-8681bf7ffb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.819099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada25c8a-7bb5-40de-9f28-8681bf7ffb3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.825802 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada25c8a-7bb5-40de-9f28-8681bf7ffb3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.827222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada25c8a-7bb5-40de-9f28-8681bf7ffb3f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ada25c8a-7bb5-40de-9f28-8681bf7ffb3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:39.841597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:39.842500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:39.843382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:39.843382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:39.843382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:39.843382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:39.843382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:39.843382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:39.843382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:39.843382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:39.843382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 522/1040] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:39.848578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada25c8a-7bb5-40de-9f28-8681bf7ffb3f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.894715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84d157-a948-4341-ae79-33c026638ddd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.931161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56cfae-8559-438b-b195-3bbb4a10a15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.931977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56cfae-8559-438b-b195-3bbb4a10a15d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.932304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56cfae-8559-438b-b195-3bbb4a10a15d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.932584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56cfae-8559-438b-b195-3bbb4a10a15d None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.933483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56cfae-8559-438b-b195-3bbb4a10a15d None admin] RBAC: Adding query filter params (type=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:39.933794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56cfae-8559-438b-b195-3bbb4a10a15d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.935185 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56cfae-8559-438b-b195-3bbb4a10a15d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.946108 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 519/1041] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:39.947165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56cfae-8559-438b-b195-3bbb4a10a15d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.963566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada25c8a-7bb5-40de-9f28-8681bf7ffb3f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.968675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:39.991287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:40.003286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d0635-348e-46cd-90c9-fdba7397fd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:40.004119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d0635-348e-46cd-90c9-fdba7397fd01 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.004280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d0635-348e-46cd-90c9-fdba7397fd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.004561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d0635-348e-46cd-90c9-fdba7397fd01 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.005400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d0635-348e-46cd-90c9-fdba7397fd01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:40.005718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d0635-348e-46cd-90c9-fdba7397fd01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:40.007139 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d0635-348e-46cd-90c9-fdba7397fd01 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.019495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 523/1042] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:40.025309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d0635-348e-46cd-90c9-fdba7397fd01 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:40.060559 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56cfae-8559-438b-b195-3bbb4a10a15d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:40.102327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b412cb9-bcf3-4344-9da8-496088d9e814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:40.103092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b412cb9-bcf3-4344-9da8-496088d9e814 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.103288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b412cb9-bcf3-4344-9da8-496088d9e814 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.103567 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b412cb9-bcf3-4344-9da8-496088d9e814 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.104397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b412cb9-bcf3-4344-9da8-496088d9e814 None admin] RBAC: Adding query filter params (interface=public, service_id=1c4badb6f9ef4766afb3b55db62c2a20, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:40.104711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b412cb9-bcf3-4344-9da8-496088d9e814 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:40.106626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b412cb9-bcf3-4344-9da8-496088d9e814 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.112569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 520/1043] 10.222.0.65 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:39 2022] GET /identity/v3/endpoints?service_id=1c4badb6f9ef4766afb3b55db62c2a20&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:40.117986 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b412cb9-bcf3-4344-9da8-496088d9e814 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:40.136377 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d0635-348e-46cd-90c9-fdba7397fd01 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:40.173577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c73e16-6aeb-4bde-8367-67ab9795cae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:40.174229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c73e16-6aeb-4bde-8367-67ab9795cae7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.174519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c73e16-6aeb-4bde-8367-67ab9795cae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.174857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c73e16-6aeb-4bde-8367-67ab9795cae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.175737 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c73e16-6aeb-4bde-8367-67ab9795cae7 None admin] RBAC: Adding query filter params (role.id=7218997909e546eaa53089a27c7886e9, scope.project.id=8458544c021049c587856343e8001429, user.id=b01d875111bd43698bcf99d30f7a7f9a) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:40.176057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c73e16-6aeb-4bde-8367-67ab9795cae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:40.177473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c73e16-6aeb-4bde-8367-67ab9795cae7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.191698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 524/1044] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:40 2022] GET /identity/v3/role_assignments?user.id=b01d875111bd43698bcf99d30f7a7f9a&scope.project.id=8458544c021049c587856343e8001429&role.id=7218997909e546eaa53089a27c7886e9 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:40.211533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:40.233008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b412cb9-bcf3-4344-9da8-496088d9e814 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:40.270211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59e938-44f2-432c-a545-1870dff828a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:40.270889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59e938-44f2-432c-a545-1870dff828a2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.271168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59e938-44f2-432c-a545-1870dff828a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.271442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59e938-44f2-432c-a545-1870dff828a2 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.272162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59e938-44f2-432c-a545-1870dff828a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:40.273546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59e938-44f2-432c-a545-1870dff828a2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.281425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 521/1045] 10.222.0.65 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:40 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 10:50:41.245445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fbaef-6397-44cf-9a7e-ec6e409ea3f0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.245945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fbaef-6397-44cf-9a7e-ec6e409ea3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.246023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fbaef-6397-44cf-9a7e-ec6e409ea3f0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.248062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 525/1046] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:41.252966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6b961-de15-46ae-ba72-5ced62ee33a9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.253378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6b961-de15-46ae-ba72-5ced62ee33a9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.253551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6b961-de15-46ae-ba72-5ced62ee33a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.271580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-84a6b961-de15-46ae-ba72-5ced62ee33a9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:41.272810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-84a6b961-de15-46ae-ba72-5ced62ee33a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:41.343699 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 522/1047] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:41.365374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fbaef-6397-44cf-9a7e-ec6e409ea3f0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:41.419992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:41.470941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fbaef-6397-44cf-9a7e-ec6e409ea3f0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:41.504664 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360dc51-1517-4d09-b0c7-71314152722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:41.505451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360dc51-1517-4d09-b0c7-71314152722c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.505733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360dc51-1517-4d09-b0c7-71314152722c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.506007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360dc51-1517-4d09-b0c7-71314152722c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.515916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360dc51-1517-4d09-b0c7-71314152722c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1360dc51-1517-4d09-b0c7-71314152722c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:41.523376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:41.524527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:41.525401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:41.525401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:41.525401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:41.525401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:41.525401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:41.525401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:41.525401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:41.525401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:41.525401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:41.525401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:41.525401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:41.525401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:41.526603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 526/1048] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:41.530518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6b961-de15-46ae-ba72-5ced62ee33a9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:41.609191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:41.633889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6b961-de15-46ae-ba72-5ced62ee33a9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:41.667476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad6c15-5935-4f28-bc1a-ecb0088199f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:41.668187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad6c15-5935-4f28-bc1a-ecb0088199f4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.669003 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad6c15-5935-4f28-bc1a-ecb0088199f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.669295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad6c15-5935-4f28-bc1a-ecb0088199f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.670038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad6c15-5935-4f28-bc1a-ecb0088199f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:41.670347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad6c15-5935-4f28-bc1a-ecb0088199f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:41.671732 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad6c15-5935-4f28-bc1a-ecb0088199f4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:41.685357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 523/1049] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:41.686289 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360dc51-1517-4d09-b0c7-71314152722c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:41.788327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360dc51-1517-4d09-b0c7-71314152722c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:41.799240 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:41.823151 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e8ea7-fdfb-41bb-9910-86cba2f97d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:41.824049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e8ea7-fdfb-41bb-9910-86cba2f97d30 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.824329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e8ea7-fdfb-41bb-9910-86cba2f97d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.824599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e8ea7-fdfb-41bb-9910-86cba2f97d30 None admin] PATH_INFO: `/v3/users/d543826da04e4f0598ab33ea021560af` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.837788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e8ea7-fdfb-41bb-9910-86cba2f97d30 None admin] RBAC: Authorizing `identity:get_user(user_id=d543826da04e4f0598ab33ea021560af)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:41.839219 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e8ea7-fdfb-41bb-9910-86cba2f97d30 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:41.846050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 527/1050] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:50:41 2022] GET /identity/v3/users/d543826da04e4f0598ab33ea021560af => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:41.847214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad6c15-5935-4f28-bc1a-ecb0088199f4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:41.948009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad6c15-5935-4f28-bc1a-ecb0088199f4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:41.981776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ad794-1bcc-4924-8aa0-8ce825ea3837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:41.982430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ad794-1bcc-4924-8aa0-8ce825ea3837 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.982710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ad794-1bcc-4924-8aa0-8ce825ea3837 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.982984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ad794-1bcc-4924-8aa0-8ce825ea3837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.991821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:42.051245 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ad794-1bcc-4924-8aa0-8ce825ea3837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.052732 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ad794-1bcc-4924-8aa0-8ce825ea3837 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.154062 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 524/1051] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 10:50:42.159996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e8ea7-fdfb-41bb-9910-86cba2f97d30 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.195789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199c2fa-167d-4c87-9acc-b29c22d891dd None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.196082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199c2fa-167d-4c87-9acc-b29c22d891dd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.196364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199c2fa-167d-4c87-9acc-b29c22d891dd None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.198980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 525/1052] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:42.203455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e1113-7019-49b8-83d9-87425d429283 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.203779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e1113-7019-49b8-83d9-87425d429283 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.204060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e1113-7019-49b8-83d9-87425d429283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.221010 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2a6e1113-7019-49b8-83d9-87425d429283 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:42.222193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2a6e1113-7019-49b8-83d9-87425d429283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:42.251853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:42.261426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e8ea7-fdfb-41bb-9910-86cba2f97d30 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.291239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 526/1053] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:42.297328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40472dc5-e214-49e9-ba94-0b78e9394aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.298188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40472dc5-e214-49e9-ba94-0b78e9394aae None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.298470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40472dc5-e214-49e9-ba94-0b78e9394aae None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.298743 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40472dc5-e214-49e9-ba94-0b78e9394aae None admin] PATH_INFO: `/v3/projects/46833a9f550143588c1c1675edf4615e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.299922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40472dc5-e214-49e9-ba94-0b78e9394aae None admin] RBAC: Authorizing `identity:get_project(project_id=46833a9f550143588c1c1675edf4615e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.301328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40472dc5-e214-49e9-ba94-0b78e9394aae None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.308118 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 528/1054] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/projects/46833a9f550143588c1c1675edf4615e => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:42.308973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e1113-7019-49b8-83d9-87425d429283 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.313886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40472dc5-e214-49e9-ba94-0b78e9394aae None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.348244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:42.408831 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e1113-7019-49b8-83d9-87425d429283 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.412932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40472dc5-e214-49e9-ba94-0b78e9394aae None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.440836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188f1b5-cb78-4f8c-874f-2bf2ce0bfcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.441485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188f1b5-cb78-4f8c-874f-2bf2ce0bfcab None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.441763 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188f1b5-cb78-4f8c-874f-2bf2ce0bfcab None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.442035 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188f1b5-cb78-4f8c-874f-2bf2ce0bfcab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.442859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188f1b5-cb78-4f8c-874f-2bf2ce0bfcab None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=46833a9f550143588c1c1675edf4615e, user.id=d543826da04e4f0598ab33ea021560af) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:42.443170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188f1b5-cb78-4f8c-874f-2bf2ce0bfcab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.444632 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188f1b5-cb78-4f8c-874f-2bf2ce0bfcab None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.445815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aef1c-eb6d-4758-bf9b-4db4d98f1e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.446621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aef1c-eb6d-4758-bf9b-4db4d98f1e81 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.446904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aef1c-eb6d-4758-bf9b-4db4d98f1e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.447511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aef1c-eb6d-4758-bf9b-4db4d98f1e81 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.450527 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 527/1055] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/role_assignments?user.id=d543826da04e4f0598ab33ea021560af&scope.project.id=46833a9f550143588c1c1675edf4615e&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:42.451675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0d4bf-17c8-483f-9309-7c75347bab7e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.451971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0d4bf-17c8-483f-9309-7c75347bab7e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.452243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0d4bf-17c8-483f-9309-7c75347bab7e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.453476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 528/1056] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:42.461616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53118a-2349-45fa-b37e-57a9f7c2cd69 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.461902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53118a-2349-45fa-b37e-57a9f7c2cd69 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.462185 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53118a-2349-45fa-b37e-57a9f7c2cd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.465175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9aef1c-eb6d-4758-bf9b-4db4d98f1e81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5f9aef1c-eb6d-4758-bf9b-4db4d98f1e81 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:42.474488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:42.475512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:42.476794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:42.476794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:42.476794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:42.476794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:42.476794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:42.476794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:42.476794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:42.476794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:42.476794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:42.476794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:42.476794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:42.476794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:42.479991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 529/1057] 10.222.0.65 () {56 vars in 1120 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:42.480857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-be53118a-2349-45fa-b37e-57a9f7c2cd69 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:42.482092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-be53118a-2349-45fa-b37e-57a9f7c2cd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:42.483217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aef1c-eb6d-4758-bf9b-4db4d98f1e81 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.511427 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:42.561029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 529/1058] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:42.585420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53118a-2349-45fa-b37e-57a9f7c2cd69 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.591113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:42.598238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aef1c-eb6d-4758-bf9b-4db4d98f1e81 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.637103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ce89a-5494-40e5-9c50-107098a62e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.638619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ce89a-5494-40e5-9c50-107098a62e71 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.639095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ce89a-5494-40e5-9c50-107098a62e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.639543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ce89a-5494-40e5-9c50-107098a62e71 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.640501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ce89a-5494-40e5-9c50-107098a62e71 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:42.640982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ce89a-5494-40e5-9c50-107098a62e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.642508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ce89a-5494-40e5-9c50-107098a62e71 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.652454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 530/1059] 10.222.0.65 () {56 vars in 1115 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:42.658154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ce89a-5494-40e5-9c50-107098a62e71 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.696115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53118a-2349-45fa-b37e-57a9f7c2cd69 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.731789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d58bf6-8749-4fef-ab09-db38ccb5ae70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.732501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d58bf6-8749-4fef-ab09-db38ccb5ae70 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.732776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d58bf6-8749-4fef-ab09-db38ccb5ae70 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.733046 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d58bf6-8749-4fef-ab09-db38ccb5ae70 None admin] PATH_INFO: `/v3/services/image` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.733970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d58bf6-8749-4fef-ab09-db38ccb5ae70 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.735329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d58bf6-8749-4fef-ab09-db38ccb5ae70 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-57d58bf6-8749-4fef-ab09-db38ccb5ae70 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:42.747766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:42.748793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:42.749626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 530/1060] 10.222.0.65 () {56 vars in 1105 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:42.754601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d58bf6-8749-4fef-ab09-db38ccb5ae70 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.766950 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ce89a-5494-40e5-9c50-107098a62e71 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.783163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:42.801756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a82c6-e567-4153-b1fb-7f53c57a3d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.802731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a82c6-e567-4153-b1fb-7f53c57a3d14 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.803021 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a82c6-e567-4153-b1fb-7f53c57a3d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.803296 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a82c6-e567-4153-b1fb-7f53c57a3d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.814668 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a82c6-e567-4153-b1fb-7f53c57a3d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.816153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a82c6-e567-4153-b1fb-7f53c57a3d14 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e52a82c6-e567-4153-b1fb-7f53c57a3d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:42.834405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:42.835316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:42.836909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:42.836909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:42.836909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:42.836909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:42.836909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 531/1061] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:42.841134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a82c6-e567-4153-b1fb-7f53c57a3d14 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.854963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:42.865818 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d58bf6-8749-4fef-ab09-db38ccb5ae70 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.904852 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e8e59-4660-4567-a6ab-263f266cd877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.905533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e8e59-4660-4567-a6ab-263f266cd877 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.905820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e8e59-4660-4567-a6ab-263f266cd877 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.906100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e8e59-4660-4567-a6ab-263f266cd877 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.906808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e8e59-4660-4567-a6ab-263f266cd877 None admin] RBAC: Adding query filter params (name=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:42.907117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e8e59-4660-4567-a6ab-263f266cd877 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.908532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e8e59-4660-4567-a6ab-263f266cd877 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.913913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 531/1062] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:42.919887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e8e59-4660-4567-a6ab-263f266cd877 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.952514 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a82c6-e567-4153-b1fb-7f53c57a3d14 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.987492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed2444-ed19-4fb6-869d-5106026ce1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.988111 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed2444-ed19-4fb6-869d-5106026ce1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.988399 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed2444-ed19-4fb6-869d-5106026ce1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.988677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed2444-ed19-4fb6-869d-5106026ce1f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.989402 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed2444-ed19-4fb6-869d-5106026ce1f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:42.989711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed2444-ed19-4fb6-869d-5106026ce1f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.991078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed2444-ed19-4fb6-869d-5106026ce1f0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:43.008266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 532/1063] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:43.009843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed2444-ed19-4fb6-869d-5106026ce1f0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:43.016424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:43.028112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e8e59-4660-4567-a6ab-263f266cd877 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:43.063900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe487f-a03c-458e-8665-c3f2988d202a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:43.064569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe487f-a03c-458e-8665-c3f2988d202a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:43.064853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe487f-a03c-458e-8665-c3f2988d202a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:43.065135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe487f-a03c-458e-8665-c3f2988d202a None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:43.065838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe487f-a03c-458e-8665-c3f2988d202a None admin] RBAC: Adding query filter params (type=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:43.066148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe487f-a03c-458e-8665-c3f2988d202a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:43.067513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe487f-a03c-458e-8665-c3f2988d202a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:43.072949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 532/1064] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:43.078578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe487f-a03c-458e-8665-c3f2988d202a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:43.094317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:43.129210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed2444-ed19-4fb6-869d-5106026ce1f0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:43.164761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45528a-9679-48fe-8fb1-99842c8ca2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:43.165565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45528a-9679-48fe-8fb1-99842c8ca2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:43.165850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45528a-9679-48fe-8fb1-99842c8ca2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:43.166129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45528a-9679-48fe-8fb1-99842c8ca2d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:43.174114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45528a-9679-48fe-8fb1-99842c8ca2d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:43.175539 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45528a-9679-48fe-8fb1-99842c8ca2d0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:43.189549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe487f-a03c-458e-8665-c3f2988d202a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-bf45528a-9679-48fe-8fb1-99842c8ca2d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:43.189838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:43.190770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:43.191768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:43.191768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:43.191768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:43.191768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:43.191768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:43.191768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:43.191768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:43.191768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:43.191768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:43.191768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:43.191768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 533/1065] 10.222.0.65 () {56 vars in 1093 bytes} [Thu Jun 23 10:50:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:43.196700 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45528a-9679-48fe-8fb1-99842c8ca2d0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:43.226863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4601b16-9fdd-4005-8eb0-5f926015b58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:43.227530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4601b16-9fdd-4005-8eb0-5f926015b58f None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:43.228023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4601b16-9fdd-4005-8eb0-5f926015b58f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:43.228132 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4601b16-9fdd-4005-8eb0-5f926015b58f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:43.228848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4601b16-9fdd-4005-8eb0-5f926015b58f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:43.230212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4601b16-9fdd-4005-8eb0-5f926015b58f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:43.251120 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 533/1066] 10.222.0.65 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:50:43.317055 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45528a-9679-48fe-8fb1-99842c8ca2d0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:43.348953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:43.363029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199d69a-594b-4102-93ce-dbe3b8ed8037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:43.363746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199d69a-594b-4102-93ce-dbe3b8ed8037 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:43.364034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199d69a-594b-4102-93ce-dbe3b8ed8037 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:43.364307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199d69a-594b-4102-93ce-dbe3b8ed8037 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:43.365186 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199d69a-594b-4102-93ce-dbe3b8ed8037 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:43.365501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199d69a-594b-4102-93ce-dbe3b8ed8037 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:43.366944 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199d69a-594b-4102-93ce-dbe3b8ed8037 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:43.391534 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 534/1067] 10.222.0.65 () {56 vars in 1133 bytes} [Thu Jun 23 10:50:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:43.393060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4601b16-9fdd-4005-8eb0-5f926015b58f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:43.406815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:43.498873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4601b16-9fdd-4005-8eb0-5f926015b58f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:43.537531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3aa2f-7058-4992-b925-c5ab5f348632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:43.538352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3aa2f-7058-4992-b925-c5ab5f348632 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:43.538636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3aa2f-7058-4992-b925-c5ab5f348632 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:43.538916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3aa2f-7058-4992-b925-c5ab5f348632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:43.551096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3aa2f-7058-4992-b925-c5ab5f348632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:43.552608 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3aa2f-7058-4992-b925-c5ab5f348632 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0ea3aa2f-7058-4992-b925-c5ab5f348632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:43.570533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:43.571599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:43.572707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:43.572707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:43.572707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:43.572707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:43.572707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 534/1068] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:43.579243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199d69a-594b-4102-93ce-dbe3b8ed8037 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:43.679434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199d69a-594b-4102-93ce-dbe3b8ed8037 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:43.713128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb2cdb-f582-4ab7-a395-fcde9981bd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:43.713800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb2cdb-f582-4ab7-a395-fcde9981bd22 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:43.714085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb2cdb-f582-4ab7-a395-fcde9981bd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:43.714361 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb2cdb-f582-4ab7-a395-fcde9981bd22 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:43.715081 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb2cdb-f582-4ab7-a395-fcde9981bd22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:43.715417 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb2cdb-f582-4ab7-a395-fcde9981bd22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:43.716815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb2cdb-f582-4ab7-a395-fcde9981bd22 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:43.732946 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 535/1069] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:43.734422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3aa2f-7058-4992-b925-c5ab5f348632 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:43.796556 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:43.837022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3aa2f-7058-4992-b925-c5ab5f348632 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:43.870653 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f2ab2-d355-4bce-a288-c14185f5f378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:43.871451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f2ab2-d355-4bce-a288-c14185f5f378 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:43.872367 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f2ab2-d355-4bce-a288-c14185f5f378 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:43.872647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f2ab2-d355-4bce-a288-c14185f5f378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:43.879230 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f2ab2-d355-4bce-a288-c14185f5f378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:43.880685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f2ab2-d355-4bce-a288-c14185f5f378 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-fd3f2ab2-d355-4bce-a288-c14185f5f378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:43.895450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:43.896399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:43.897313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:43.897313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:43.897313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:43.897313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:43.897313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:43.897313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:43.897313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:43.897313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:43.898472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 535/1070] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:43.902619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb2cdb-f582-4ab7-a395-fcde9981bd22 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:43.951085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:44.010835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb2cdb-f582-4ab7-a395-fcde9981bd22 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:44.045816 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88ef74-119b-42b8-89b8-cdcc1e3d765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:44.046477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88ef74-119b-42b8-89b8-cdcc1e3d765f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.046812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88ef74-119b-42b8-89b8-cdcc1e3d765f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.047029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88ef74-119b-42b8-89b8-cdcc1e3d765f None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.047825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88ef74-119b-42b8-89b8-cdcc1e3d765f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:44.048191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88ef74-119b-42b8-89b8-cdcc1e3d765f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:44.049599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88ef74-119b-42b8-89b8-cdcc1e3d765f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:44.067183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 536/1071] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:44.068764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f2ab2-d355-4bce-a288-c14185f5f378 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:44.173993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f2ab2-d355-4bce-a288-c14185f5f378 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:44.196527 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:44.209942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1f3b4-fc3c-416a-8f00-548e9f87f7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:44.210824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1f3b4-fc3c-416a-8f00-548e9f87f7dc None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.211101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1f3b4-fc3c-416a-8f00-548e9f87f7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.211372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1f3b4-fc3c-416a-8f00-548e9f87f7dc None admin] PATH_INFO: `/v3/projects/8458544c021049c587856343e8001429/users/b01d875111bd43698bcf99d30f7a7f9a/roles/7218997909e546eaa53089a27c7886e9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.213551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1f3b4-fc3c-416a-8f00-548e9f87f7dc None admin] RBAC: Authorizing `identity:create_grant(project_id=8458544c021049c587856343e8001429, user_id=b01d875111bd43698bcf99d30f7a7f9a, role_id=7218997909e546eaa53089a27c7886e9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:44.214969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1f3b4-fc3c-416a-8f00-548e9f87f7dc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:44.229847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 536/1072] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:44 2022] PUT /identity/v3/projects/8458544c021049c587856343e8001429/users/b01d875111bd43698bcf99d30f7a7f9a/roles/7218997909e546eaa53089a27c7886e9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:44.487144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1e645-aedb-4714-80ba-22abbe55a85d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.487648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1e645-aedb-4714-80ba-22abbe55a85d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.487648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1e645-aedb-4714-80ba-22abbe55a85d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.489153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 537/1073] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:44.494188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee01640-a000-4ca8-a29b-236196cb85d0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.494563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee01640-a000-4ca8-a29b-236196cb85d0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.494748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee01640-a000-4ca8-a29b-236196cb85d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.511923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9ee01640-a000-4ca8-a29b-236196cb85d0 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:44.513149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9ee01640-a000-4ca8-a29b-236196cb85d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:44.582476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 537/1074] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:44.602136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1e645-aedb-4714-80ba-22abbe55a85d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:44.703783 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1e645-aedb-4714-80ba-22abbe55a85d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:44.706548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:44.740034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b59a6e-283c-41ba-9b75-f4ea90cc8894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:44.740753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b59a6e-283c-41ba-9b75-f4ea90cc8894 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.741039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b59a6e-283c-41ba-9b75-f4ea90cc8894 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.741320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b59a6e-283c-41ba-9b75-f4ea90cc8894 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.751649 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b59a6e-283c-41ba-9b75-f4ea90cc8894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a5b59a6e-283c-41ba-9b75-f4ea90cc8894 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:44.759277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:44.760504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:44.761342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:44.761342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:44.761342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:44.761342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:44.761342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:44.761342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:44.761342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:44.761342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:44.761342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:44.761342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:44.761342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:44.761342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:44.762529 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 538/1075] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:44.766705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee01640-a000-4ca8-a29b-236196cb85d0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:44.867165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee01640-a000-4ca8-a29b-236196cb85d0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:44.881227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:44.904181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa73796-f437-4376-b16e-d1e454061170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:44.904884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa73796-f437-4376-b16e-d1e454061170 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.905166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa73796-f437-4376-b16e-d1e454061170 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.905440 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa73796-f437-4376-b16e-d1e454061170 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.906184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa73796-f437-4376-b16e-d1e454061170 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:44.906491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa73796-f437-4376-b16e-d1e454061170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:44.907885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa73796-f437-4376-b16e-d1e454061170 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:44.921451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 538/1076] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:44.922667 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b59a6e-283c-41ba-9b75-f4ea90cc8894 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.021965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b59a6e-283c-41ba-9b75-f4ea90cc8894 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.055296 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6eb87f-dfc4-42fd-81c3-7ab213551454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:45.056185 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6eb87f-dfc4-42fd-81c3-7ab213551454 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.056467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6eb87f-dfc4-42fd-81c3-7ab213551454 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.056740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6eb87f-dfc4-42fd-81c3-7ab213551454 None admin] PATH_INFO: `/v3/users/d543826da04e4f0598ab33ea021560af` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.057903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6eb87f-dfc4-42fd-81c3-7ab213551454 None admin] RBAC: Authorizing `identity:get_user(user_id=d543826da04e4f0598ab33ea021560af)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:45.059338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6eb87f-dfc4-42fd-81c3-7ab213551454 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.061171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 539/1077] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:50:44 2022] GET /identity/v3/users/d543826da04e4f0598ab33ea021560af => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:45.066639 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa73796-f437-4376-b16e-d1e454061170 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.175927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa73796-f437-4376-b16e-d1e454061170 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.209007 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd50cd-da15-4186-99dc-0361130dbbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:45.209866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd50cd-da15-4186-99dc-0361130dbbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.210144 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd50cd-da15-4186-99dc-0361130dbbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.210574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd50cd-da15-4186-99dc-0361130dbbd0 None admin] PATH_INFO: `/v3/projects/46833a9f550143588c1c1675edf4615e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.211782 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd50cd-da15-4186-99dc-0361130dbbd0 None admin] RBAC: Authorizing `identity:get_project(project_id=46833a9f550143588c1c1675edf4615e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:45.213205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd50cd-da15-4186-99dc-0361130dbbd0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.215114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 539/1078] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:50:45 2022] GET /identity/v3/projects/46833a9f550143588c1c1675edf4615e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:45.221291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6eb87f-dfc4-42fd-81c3-7ab213551454 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.254194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:45.320199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5b676-8900-4eed-b0e7-13ef173a8478 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.320508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5b676-8900-4eed-b0e7-13ef173a8478 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.320686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5b676-8900-4eed-b0e7-13ef173a8478 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.322240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 540/1079] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:45.327323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6851cfa-2ad9-4c75-8d50-ac86eca630f9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.327618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6851cfa-2ad9-4c75-8d50-ac86eca630f9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.327942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6851cfa-2ad9-4c75-8d50-ac86eca630f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.329137 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6eb87f-dfc4-42fd-81c3-7ab213551454 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.344584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a6851cfa-2ad9-4c75-8d50-ac86eca630f9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:45.345774 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a6851cfa-2ad9-4c75-8d50-ac86eca630f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:45.369316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4f184-c2c8-423f-ae20-d7b049bb7cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:45.370217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4f184-c2c8-423f-ae20-d7b049bb7cd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.370505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4f184-c2c8-423f-ae20-d7b049bb7cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.370784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4f184-c2c8-423f-ae20-d7b049bb7cd2 None admin] PATH_INFO: `/v3/projects/46833a9f550143588c1c1675edf4615e/users/d543826da04e4f0598ab33ea021560af/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.373385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4f184-c2c8-423f-ae20-d7b049bb7cd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=46833a9f550143588c1c1675edf4615e, user_id=d543826da04e4f0598ab33ea021560af, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:45.374821 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4f184-c2c8-423f-ae20-d7b049bb7cd2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.384928 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:45.389397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 540/1080] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:45 2022] PUT /identity/v3/projects/46833a9f550143588c1c1675edf4615e/users/d543826da04e4f0598ab33ea021560af/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:50:45.421460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 541/1081] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:45.445897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4f184-c2c8-423f-ae20-d7b049bb7cd2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.549740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:45.560233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4f184-c2c8-423f-ae20-d7b049bb7cd2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.594889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f03ed-0790-4673-82db-71932b1a70fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:45.595467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f03ed-0790-4673-82db-71932b1a70fe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.595783 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f03ed-0790-4673-82db-71932b1a70fe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.596068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f03ed-0790-4673-82db-71932b1a70fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.666282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f03ed-0790-4673-82db-71932b1a70fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:45.667770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f03ed-0790-4673-82db-71932b1a70fe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.758267 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:45.773834 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 541/1082] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:45.775034 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6851cfa-2ad9-4c75-8d50-ac86eca630f9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.877560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6851cfa-2ad9-4c75-8d50-ac86eca630f9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.913044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b69d1-bb8b-45ab-b24d-ca5d1f8fd7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:45.913842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b69d1-bb8b-45ab-b24d-ca5d1f8fd7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.914128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b69d1-bb8b-45ab-b24d-ca5d1f8fd7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.914405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b69d1-bb8b-45ab-b24d-ca5d1f8fd7a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.924748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b69d1-bb8b-45ab-b24d-ca5d1f8fd7a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:45.926207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b69d1-bb8b-45ab-b24d-ca5d1f8fd7a3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6a6b69d1-bb8b-45ab-b24d-ca5d1f8fd7a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:45.943789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:45.944723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:45.951386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:45.951386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:45.951386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:45.951386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:45.951386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 542/1083] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:45.951727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f03ed-0790-4673-82db-71932b1a70fe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:46.056466 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f03ed-0790-4673-82db-71932b1a70fe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:46.098547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837658b6-0555-44b0-8dd0-1e35faa10d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:46.099234 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837658b6-0555-44b0-8dd0-1e35faa10d94 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.099522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837658b6-0555-44b0-8dd0-1e35faa10d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.099833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837658b6-0555-44b0-8dd0-1e35faa10d94 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.100577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837658b6-0555-44b0-8dd0-1e35faa10d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:46.100889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837658b6-0555-44b0-8dd0-1e35faa10d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:46.102254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837658b6-0555-44b0-8dd0-1e35faa10d94 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:46.119677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 542/1084] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:46.236639 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04691823-73a0-4b79-9971-31f0053065b9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.236826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04691823-73a0-4b79-9971-31f0053065b9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.237102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04691823-73a0-4b79-9971-31f0053065b9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.238500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 543/1085] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:46.244173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239068e4-7a1b-4f27-a665-b50b04955425 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.244462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239068e4-7a1b-4f27-a665-b50b04955425 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.244742 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239068e4-7a1b-4f27-a665-b50b04955425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.262803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-239068e4-7a1b-4f27-a665-b50b04955425 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:46.264034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-239068e4-7a1b-4f27-a665-b50b04955425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:46.313308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:46.335618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 543/1086] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:46.359198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04691823-73a0-4b79-9971-31f0053065b9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:46.386704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:46.465750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04691823-73a0-4b79-9971-31f0053065b9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:46.500420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b5175-0e35-4c0b-b5ae-07b08f43f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:46.501159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b5175-0e35-4c0b-b5ae-07b08f43f677 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.501440 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b5175-0e35-4c0b-b5ae-07b08f43f677 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.501717 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b5175-0e35-4c0b-b5ae-07b08f43f677 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.511569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b5175-0e35-4c0b-b5ae-07b08f43f677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ff7b5175-0e35-4c0b-b5ae-07b08f43f677 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:46.520282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:46.521404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:46.522278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 544/1087] 10.222.0.65 () {56 vars in 1120 bytes} [Thu Jun 23 10:50:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:46.527404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239068e4-7a1b-4f27-a665-b50b04955425 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:46.623910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239068e4-7a1b-4f27-a665-b50b04955425 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:46.655956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35ffa6-2e89-4a8e-9a81-e482da08f067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:46.656652 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35ffa6-2e89-4a8e-9a81-e482da08f067 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.656933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35ffa6-2e89-4a8e-9a81-e482da08f067 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.657205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35ffa6-2e89-4a8e-9a81-e482da08f067 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.657943 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35ffa6-2e89-4a8e-9a81-e482da08f067 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:46.658247 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35ffa6-2e89-4a8e-9a81-e482da08f067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:46.659593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35ffa6-2e89-4a8e-9a81-e482da08f067 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:46.667143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 544/1088] 10.222.0.65 () {56 vars in 1115 bytes} [Thu Jun 23 10:50:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:46.673208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b5175-0e35-4c0b-b5ae-07b08f43f677 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:46.755157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:46.771553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b5175-0e35-4c0b-b5ae-07b08f43f677 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:46.804054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2ee4e-9850-4214-bb37-4b436caa77f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:46.804914 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2ee4e-9850-4214-bb37-4b436caa77f4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.805194 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2ee4e-9850-4214-bb37-4b436caa77f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.805465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2ee4e-9850-4214-bb37-4b436caa77f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.815618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2ee4e-9850-4214-bb37-4b436caa77f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:46.817068 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2ee4e-9850-4214-bb37-4b436caa77f4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-35c2ee4e-9850-4214-bb37-4b436caa77f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:46.834077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:46.834977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:46.835924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:46.835924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:46.835924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:46.835924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:46.835924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 545/1089] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:46.841526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35ffa6-2e89-4a8e-9a81-e482da08f067 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:46.887860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:46.941637 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35ffa6-2e89-4a8e-9a81-e482da08f067 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:46.975215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a389f5-7830-4257-87cf-8724e8667c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:46.975914 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a389f5-7830-4257-87cf-8724e8667c7f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.976202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a389f5-7830-4257-87cf-8724e8667c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.976480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a389f5-7830-4257-87cf-8724e8667c7f None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.977201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a389f5-7830-4257-87cf-8724e8667c7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:46.977509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a389f5-7830-4257-87cf-8724e8667c7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:46.978900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a389f5-7830-4257-87cf-8724e8667c7f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:46.990753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 545/1090] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:46.996271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2ee4e-9850-4214-bb37-4b436caa77f4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:47.097239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2ee4e-9850-4214-bb37-4b436caa77f4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:47.130337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318142a-128c-4269-b020-000d0caa253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:47.131123 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318142a-128c-4269-b020-000d0caa253a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:47.131404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318142a-128c-4269-b020-000d0caa253a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:47.131698 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318142a-128c-4269-b020-000d0caa253a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:47.134292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:47.140570 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318142a-128c-4269-b020-000d0caa253a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:47.142195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318142a-128c-4269-b020-000d0caa253a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9318142a-128c-4269-b020-000d0caa253a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:47.155724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:47.156846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:47.162589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:47.162589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:47.162589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:47.162589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:47.162589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:47.162589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:47.162589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:47.162589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:47.162589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:47.162589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:47.162589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 546/1091] 10.222.0.65 () {56 vars in 1093 bytes} [Thu Jun 23 10:50:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:47.163727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a389f5-7830-4257-87cf-8724e8667c7f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:47.259451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a389f5-7830-4257-87cf-8724e8667c7f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:47.261770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:47.293478 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8204404-e0b4-4502-802b-3ac53ce6877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:47.294144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8204404-e0b4-4502-802b-3ac53ce6877c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:47.294423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8204404-e0b4-4502-802b-3ac53ce6877c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:47.294696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8204404-e0b4-4502-802b-3ac53ce6877c None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:47.295519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8204404-e0b4-4502-802b-3ac53ce6877c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:47.295861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8204404-e0b4-4502-802b-3ac53ce6877c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:47.297268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8204404-e0b4-4502-802b-3ac53ce6877c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:47.315125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 546/1092] 10.222.0.65 () {56 vars in 1133 bytes} [Thu Jun 23 10:50:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:47.320959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318142a-128c-4269-b020-000d0caa253a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:47.402982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e737c-d319-4b0d-b10e-09f4a6585145 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:47.403289 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e737c-d319-4b0d-b10e-09f4a6585145 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:47.403471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e737c-d319-4b0d-b10e-09f4a6585145 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:47.405129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 547/1093] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:47.409853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f1f26-01be-4cf4-8c44-54202e218e00 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:47.410144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f1f26-01be-4cf4-8c44-54202e218e00 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:47.410405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f1f26-01be-4cf4-8c44-54202e218e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:47.427521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-be2f1f26-01be-4cf4-8c44-54202e218e00 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:47.428836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-be2f1f26-01be-4cf4-8c44-54202e218e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:47.431540 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318142a-128c-4269-b020-000d0caa253a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:47.468287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5bb53-a187-43ab-a60c-65e29870f8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:47.468944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5bb53-a187-43ab-a60c-65e29870f8fa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:47.469229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5bb53-a187-43ab-a60c-65e29870f8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:47.469510 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5bb53-a187-43ab-a60c-65e29870f8fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:47.501905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 548/1094] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:47.525185 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f1f26-01be-4cf4-8c44-54202e218e00 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:47.536628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:47.545140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5bb53-a187-43ab-a60c-65e29870f8fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:47.546772 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5bb53-a187-43ab-a60c-65e29870f8fa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:47.620934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:47.633722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f1f26-01be-4cf4-8c44-54202e218e00 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:47.658223 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 547/1095] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:47.663967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5bb53-a187-43ab-a60c-65e29870f8fa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:47.667244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378ce71-21e1-4f26-accb-0c57c25ab1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:47.668072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378ce71-21e1-4f26-accb-0c57c25ab1dc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:47.668356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378ce71-21e1-4f26-accb-0c57c25ab1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:47.669162 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378ce71-21e1-4f26-accb-0c57c25ab1dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:47.679382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378ce71-21e1-4f26-accb-0c57c25ab1dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4378ce71-21e1-4f26-accb-0c57c25ab1dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:47.687524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:47.688600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:50:47.689451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 549/1096] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:47.694500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378ce71-21e1-4f26-accb-0c57c25ab1dc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:47.756460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:47.762683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5bb53-a187-43ab-a60c-65e29870f8fa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:47.792865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378ce71-21e1-4f26-accb-0c57c25ab1dc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:47.796840 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19157e57-13da-40cb-9770-8802ec3342af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:47.797707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19157e57-13da-40cb-9770-8802ec3342af None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:47.798048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19157e57-13da-40cb-9770-8802ec3342af None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:47.798388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19157e57-13da-40cb-9770-8802ec3342af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:47.808329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19157e57-13da-40cb-9770-8802ec3342af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:47.809805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19157e57-13da-40cb-9770-8802ec3342af None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-19157e57-13da-40cb-9770-8802ec3342af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:47.827169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:47.828140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:47.829057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:47.829057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:47.829057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:47.829057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:47.829057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 548/1097] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:47.831810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51f5e6-79a7-417d-a307-c06a6b59fb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:47.832484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51f5e6-79a7-417d-a307-c06a6b59fb77 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:47.832767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51f5e6-79a7-417d-a307-c06a6b59fb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:47.833067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51f5e6-79a7-417d-a307-c06a6b59fb77 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:47.833817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51f5e6-79a7-417d-a307-c06a6b59fb77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:47.834127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51f5e6-79a7-417d-a307-c06a6b59fb77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:47.834580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19157e57-13da-40cb-9770-8802ec3342af None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:47.835486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51f5e6-79a7-417d-a307-c06a6b59fb77 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:47.848936 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 550/1098] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:47.849877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51f5e6-79a7-417d-a307-c06a6b59fb77 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:47.866370 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:47.933771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19157e57-13da-40cb-9770-8802ec3342af None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:47.949500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51f5e6-79a7-417d-a307-c06a6b59fb77 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:47.966879 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172309b-7a9e-4eaf-abab-8210b449789f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:47.967540 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172309b-7a9e-4eaf-abab-8210b449789f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:47.967856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172309b-7a9e-4eaf-abab-8210b449789f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:47.968138 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172309b-7a9e-4eaf-abab-8210b449789f None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:47.968854 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172309b-7a9e-4eaf-abab-8210b449789f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:47.969169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172309b-7a9e-4eaf-abab-8210b449789f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:47.970549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172309b-7a9e-4eaf-abab-8210b449789f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:47.982081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 549/1099] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:47.983213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a50a8c-1b4b-4657-8091-2733cf4b3361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:47.984365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a50a8c-1b4b-4657-8091-2733cf4b3361 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:47.984647 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a50a8c-1b4b-4657-8091-2733cf4b3361 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:47.984918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a50a8c-1b4b-4657-8091-2733cf4b3361 None admin] PATH_INFO: `/v3/users/d543826da04e4f0598ab33ea021560af` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:47.986100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a50a8c-1b4b-4657-8091-2733cf4b3361 None admin] RBAC: Authorizing `identity:get_user(user_id=d543826da04e4f0598ab33ea021560af)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:47.987681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a50a8c-1b4b-4657-8091-2733cf4b3361 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:47.987854 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172309b-7a9e-4eaf-abab-8210b449789f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:47.989388 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 551/1100] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:50:47 2022] GET /identity/v3/users/d543826da04e4f0598ab33ea021560af => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:47.994521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a50a8c-1b4b-4657-8091-2733cf4b3361 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:48.002008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:48.074876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:48.089653 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172309b-7a9e-4eaf-abab-8210b449789f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:48.094261 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a50a8c-1b4b-4657-8091-2733cf4b3361 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:48.126073 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2d05f-dbbb-453d-a266-2c167554a2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:48.126870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2d05f-dbbb-453d-a266-2c167554a2fc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.127150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2d05f-dbbb-453d-a266-2c167554a2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.127313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd96d81-835f-423d-a3b6-e2c163214c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:48.127422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2d05f-dbbb-453d-a266-2c167554a2fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.127981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd96d81-835f-423d-a3b6-e2c163214c0f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.128264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd96d81-835f-423d-a3b6-e2c163214c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.128540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd96d81-835f-423d-a3b6-e2c163214c0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.133990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2d05f-dbbb-453d-a266-2c167554a2fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:48.135427 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2d05f-dbbb-453d-a266-2c167554a2fc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2de2d05f-dbbb-453d-a266-2c167554a2fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:48.150072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:48.151101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:48.152188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:48.152188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:48.152188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:48.152188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:48.152188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:48.152188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:48.152188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:48.152188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:50:48.152188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 550/1101] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:48.157442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2d05f-dbbb-453d-a266-2c167554a2fc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:48.194772 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd96d81-835f-423d-a3b6-e2c163214c0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:48.196173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd96d81-835f-423d-a3b6-e2c163214c0f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:48.221818 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:48.255052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2d05f-dbbb-453d-a266-2c167554a2fc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:48.276176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:48.289287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fc090-af44-4fab-bfc2-90d58ee19a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:48.290243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fc090-af44-4fab-bfc2-90d58ee19a35 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.290523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fc090-af44-4fab-bfc2-90d58ee19a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.290798 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fc090-af44-4fab-bfc2-90d58ee19a35 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.291579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fc090-af44-4fab-bfc2-90d58ee19a35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:48.291911 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fc090-af44-4fab-bfc2-90d58ee19a35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:48.293322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fc090-af44-4fab-bfc2-90d58ee19a35 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:48.300482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 552/1102] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 10:50:48.301463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd96d81-835f-423d-a3b6-e2c163214c0f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:48.309642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 551/1103] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:48.310767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fc090-af44-4fab-bfc2-90d58ee19a35 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:48.397432 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd96d81-835f-423d-a3b6-e2c163214c0f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:48.407241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fc090-af44-4fab-bfc2-90d58ee19a35 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:48.430073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0ecf1-1098-4b13-bad8-d3125cbbe69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:48.430926 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0ecf1-1098-4b13-bad8-d3125cbbe69e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.431214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0ecf1-1098-4b13-bad8-d3125cbbe69e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.431489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0ecf1-1098-4b13-bad8-d3125cbbe69e None admin] PATH_INFO: `/v3/projects/46833a9f550143588c1c1675edf4615e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.432038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:48.432694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0ecf1-1098-4b13-bad8-d3125cbbe69e None admin] RBAC: Authorizing `identity:get_project(project_id=46833a9f550143588c1c1675edf4615e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:48.434207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0ecf1-1098-4b13-bad8-d3125cbbe69e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:48.435933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 553/1104] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:50:48 2022] GET /identity/v3/projects/46833a9f550143588c1c1675edf4615e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:48.440370 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447295a-8515-4ffc-9012-7488b56ea65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:48.441288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447295a-8515-4ffc-9012-7488b56ea65e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.441572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447295a-8515-4ffc-9012-7488b56ea65e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.441804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0ecf1-1098-4b13-bad8-d3125cbbe69e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:48.441898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447295a-8515-4ffc-9012-7488b56ea65e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.442665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447295a-8515-4ffc-9012-7488b56ea65e None admin] RBAC: Adding query filter params (role.id=7218997909e546eaa53089a27c7886e9, scope.project.id=8458544c021049c587856343e8001429, user.id=b01d875111bd43698bcf99d30f7a7f9a) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:48.442980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447295a-8515-4ffc-9012-7488b56ea65e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:48.444434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447295a-8515-4ffc-9012-7488b56ea65e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:48.456177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 552/1105] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:48 2022] GET /identity/v3/role_assignments?user.id=b01d875111bd43698bcf99d30f7a7f9a&scope.project.id=8458544c021049c587856343e8001429&role.id=7218997909e546eaa53089a27c7886e9 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:48.494854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:48.548526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0ecf1-1098-4b13-bad8-d3125cbbe69e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:48.584709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d992f-5c66-420d-b18f-5464f6119fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:48.585381 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d992f-5c66-420d-b18f-5464f6119fcc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.585665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d992f-5c66-420d-b18f-5464f6119fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.585942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d992f-5c66-420d-b18f-5464f6119fcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.586778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d992f-5c66-420d-b18f-5464f6119fcc None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=46833a9f550143588c1c1675edf4615e, user.id=d543826da04e4f0598ab33ea021560af) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:48.587093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d992f-5c66-420d-b18f-5464f6119fcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:48.588549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d992f-5c66-420d-b18f-5464f6119fcc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:48.596599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 554/1106] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:48 2022] GET /identity/v3/role_assignments?user.id=d543826da04e4f0598ab33ea021560af&scope.project.id=46833a9f550143588c1c1675edf4615e&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:50.616170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ed83c-d63f-468c-b9c5-acef1ddbf21d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:50.616726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ed83c-d63f-468c-b9c5-acef1ddbf21d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:50.616726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ed83c-d63f-468c-b9c5-acef1ddbf21d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:50.618113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 553/1107] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:50.623707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a58f93-c66c-4dfd-8609-6ecf6e86f38a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:50.624096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a58f93-c66c-4dfd-8609-6ecf6e86f38a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:50.624264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a58f93-c66c-4dfd-8609-6ecf6e86f38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:50.640435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-81a58f93-c66c-4dfd-8609-6ecf6e86f38a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:50.641592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-81a58f93-c66c-4dfd-8609-6ecf6e86f38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:50.704576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 555/1108] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:50.728343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ed83c-d63f-468c-b9c5-acef1ddbf21d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:50.827648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ed83c-d63f-468c-b9c5-acef1ddbf21d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:50.861215 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c11f3-3af7-40d0-a68b-d2199e1121f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:50.861991 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c11f3-3af7-40d0-a68b-d2199e1121f9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:50.862278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c11f3-3af7-40d0-a68b-d2199e1121f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:50.862557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c11f3-3af7-40d0-a68b-d2199e1121f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:50.863744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c11f3-3af7-40d0-a68b-d2199e1121f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:50.865172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c11f3-3af7-40d0-a68b-d2199e1121f9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:50.866886 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 554/1109] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:50.872615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a58f93-c66c-4dfd-8609-6ecf6e86f38a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:50.916192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:50.969046 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a58f93-c66c-4dfd-8609-6ecf6e86f38a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:51.002296 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38174835-7973-492c-bd07-d77404fc1a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:51.002954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38174835-7973-492c-bd07-d77404fc1a90 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:51.003239 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38174835-7973-492c-bd07-d77404fc1a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:51.003518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38174835-7973-492c-bd07-d77404fc1a90 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:51.004246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38174835-7973-492c-bd07-d77404fc1a90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:51.005665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38174835-7973-492c-bd07-d77404fc1a90 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:51.024978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 556/1110] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:50:50 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:53.044758 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa06fe0-1249-42fb-816f-0b50626bd697 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:53.045289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa06fe0-1249-42fb-816f-0b50626bd697 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:53.045289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa06fe0-1249-42fb-816f-0b50626bd697 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:53.046683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 555/1111] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:53.051717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e81051-28f2-4df0-9ea1-e2ad5010bc6e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:53.052155 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e81051-28f2-4df0-9ea1-e2ad5010bc6e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:53.052276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e81051-28f2-4df0-9ea1-e2ad5010bc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:53.068434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-17e81051-28f2-4df0-9ea1-e2ad5010bc6e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:53.069594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-17e81051-28f2-4df0-9ea1-e2ad5010bc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:53.136151 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 557/1112] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:53.159507 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa06fe0-1249-42fb-816f-0b50626bd697 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:53.217357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:53.257640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa06fe0-1249-42fb-816f-0b50626bd697 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:53.289983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a661139-e9d8-4729-8695-c0979dda2434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:53.290819 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a661139-e9d8-4729-8695-c0979dda2434 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:53.291101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a661139-e9d8-4729-8695-c0979dda2434 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:53.291377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a661139-e9d8-4729-8695-c0979dda2434 None admin] PATH_INFO: `/v3/roles/8a0cfaebc67a497a98459da3f1a82ef9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:53.292942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a661139-e9d8-4729-8695-c0979dda2434 None admin] RBAC: Authorizing `identity:get_role(role_id=8a0cfaebc67a497a98459da3f1a82ef9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:53.294924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a661139-e9d8-4729-8695-c0979dda2434 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:53.296254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 556/1113] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:50:53 2022] GET /identity/v3/roles/8a0cfaebc67a497a98459da3f1a82ef9 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:53.301784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e81051-28f2-4df0-9ea1-e2ad5010bc6e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:53.329919 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:53.399497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e81051-28f2-4df0-9ea1-e2ad5010bc6e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:53.432154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96311109-b9b8-4e84-b8c4-dcaf11372b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:53.433014 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96311109-b9b8-4e84-b8c4-dcaf11372b6f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:53.433299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96311109-b9b8-4e84-b8c4-dcaf11372b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:53.433577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96311109-b9b8-4e84-b8c4-dcaf11372b6f None admin] PATH_INFO: `/v3/users/d0f4a24f40524bc9b4acd1e7acd7d218` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:53.446156 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96311109-b9b8-4e84-b8c4-dcaf11372b6f None admin] RBAC: Authorizing `identity:get_user(user_id=d0f4a24f40524bc9b4acd1e7acd7d218)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:53.447593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96311109-b9b8-4e84-b8c4-dcaf11372b6f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:53.449242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 558/1114] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:50:53 2022] GET /identity/v3/users/d0f4a24f40524bc9b4acd1e7acd7d218 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:53.455323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a661139-e9d8-4729-8695-c0979dda2434 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:53.550506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a661139-e9d8-4729-8695-c0979dda2434 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:53.575456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:53.583366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b46a0c-5fce-4abc-a76d-5e593629c023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:53.584029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b46a0c-5fce-4abc-a76d-5e593629c023 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:53.584311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b46a0c-5fce-4abc-a76d-5e593629c023 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:53.584582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b46a0c-5fce-4abc-a76d-5e593629c023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:53.648142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b46a0c-5fce-4abc-a76d-5e593629c023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:53.649563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b46a0c-5fce-4abc-a76d-5e593629c023 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:53.744589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 557/1115] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:53.750832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96311109-b9b8-4e84-b8c4-dcaf11372b6f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:53.834575 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:53.846536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96311109-b9b8-4e84-b8c4-dcaf11372b6f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:53.881639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784c475-78cf-4bdc-9500-c42acbdbcb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:53.882485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784c475-78cf-4bdc-9500-c42acbdbcb03 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:53.882765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784c475-78cf-4bdc-9500-c42acbdbcb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:53.883039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784c475-78cf-4bdc-9500-c42acbdbcb03 None admin] PATH_INFO: `/v3/projects/46833a9f550143588c1c1675edf4615e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:53.884220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784c475-78cf-4bdc-9500-c42acbdbcb03 None admin] RBAC: Authorizing `identity:get_project(project_id=46833a9f550143588c1c1675edf4615e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:53.885672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784c475-78cf-4bdc-9500-c42acbdbcb03 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:53.887941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 559/1116] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:50:53 2022] GET /identity/v3/projects/46833a9f550143588c1c1675edf4615e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:53.894039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b46a0c-5fce-4abc-a76d-5e593629c023 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:53.920182 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:53.990462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b46a0c-5fce-4abc-a76d-5e593629c023 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:54.023900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d66144-01b4-4ef7-8813-b8b225f9e729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:54.024545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d66144-01b4-4ef7-8813-b8b225f9e729 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:54.024827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d66144-01b4-4ef7-8813-b8b225f9e729 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:54.025106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d66144-01b4-4ef7-8813-b8b225f9e729 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:54.025926 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d66144-01b4-4ef7-8813-b8b225f9e729 None admin] RBAC: Adding query filter params (role.id=8a0cfaebc67a497a98459da3f1a82ef9, scope.project.id=46833a9f550143588c1c1675edf4615e, user.id=d0f4a24f40524bc9b4acd1e7acd7d218) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:54.026240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d66144-01b4-4ef7-8813-b8b225f9e729 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:54.027686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d66144-01b4-4ef7-8813-b8b225f9e729 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:54.033010 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 558/1117] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:53 2022] GET /identity/v3/role_assignments?user.id=d0f4a24f40524bc9b4acd1e7acd7d218&scope.project.id=46833a9f550143588c1c1675edf4615e&role.id=8a0cfaebc67a497a98459da3f1a82ef9 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:56.012659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c858ec-1ca2-45df-bbc9-7639c0b93eda None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.013191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c858ec-1ca2-45df-bbc9-7639c0b93eda None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.013307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c858ec-1ca2-45df-bbc9-7639c0b93eda None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.014578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 560/1118] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:56.019789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6efaf-928f-473e-94a9-f62d64e428fd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.020181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6efaf-928f-473e-94a9-f62d64e428fd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.020341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6efaf-928f-473e-94a9-f62d64e428fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.036630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d3b6efaf-928f-473e-94a9-f62d64e428fd None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:56.037803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d3b6efaf-928f-473e-94a9-f62d64e428fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:56.101716 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 559/1119] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:56.125198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c858ec-1ca2-45df-bbc9-7639c0b93eda None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:56.221446 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c858ec-1ca2-45df-bbc9-7639c0b93eda None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:56.253841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30d3f0-09a8-4ddd-9786-6d6b004b0f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:56.254679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30d3f0-09a8-4ddd-9786-6d6b004b0f38 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.254958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30d3f0-09a8-4ddd-9786-6d6b004b0f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.255235 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30d3f0-09a8-4ddd-9786-6d6b004b0f38 None admin] PATH_INFO: `/v3/roles/8a0cfaebc67a497a98459da3f1a82ef9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.256697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30d3f0-09a8-4ddd-9786-6d6b004b0f38 None admin] RBAC: Authorizing `identity:get_role(role_id=8a0cfaebc67a497a98459da3f1a82ef9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:56.258034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30d3f0-09a8-4ddd-9786-6d6b004b0f38 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:56.259345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 561/1120] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:50:56 2022] GET /identity/v3/roles/8a0cfaebc67a497a98459da3f1a82ef9 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:56.265514 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6efaf-928f-473e-94a9-f62d64e428fd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:56.288191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:56.361300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6efaf-928f-473e-94a9-f62d64e428fd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:56.393542 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed660ee-fc6f-44dd-838c-85ad4904f475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:56.394388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed660ee-fc6f-44dd-838c-85ad4904f475 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.394667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed660ee-fc6f-44dd-838c-85ad4904f475 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.394941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed660ee-fc6f-44dd-838c-85ad4904f475 None admin] PATH_INFO: `/v3/users/d0f4a24f40524bc9b4acd1e7acd7d218` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.396141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed660ee-fc6f-44dd-838c-85ad4904f475 None admin] RBAC: Authorizing `identity:get_user(user_id=d0f4a24f40524bc9b4acd1e7acd7d218)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:56.397566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed660ee-fc6f-44dd-838c-85ad4904f475 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:56.399167 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 560/1121] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:50:56 2022] GET /identity/v3/users/d0f4a24f40524bc9b4acd1e7acd7d218 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:56.404785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30d3f0-09a8-4ddd-9786-6d6b004b0f38 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:56.429409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:56.501133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30d3f0-09a8-4ddd-9786-6d6b004b0f38 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:56.532839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2251e8-a529-493a-8ec3-915a37dc8dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:56.533607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2251e8-a529-493a-8ec3-915a37dc8dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.533886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2251e8-a529-493a-8ec3-915a37dc8dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.534159 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2251e8-a529-493a-8ec3-915a37dc8dd9 None admin] PATH_INFO: `/v3/projects/46833a9f550143588c1c1675edf4615e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.535314 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2251e8-a529-493a-8ec3-915a37dc8dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=46833a9f550143588c1c1675edf4615e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:56.536742 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2251e8-a529-493a-8ec3-915a37dc8dd9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:56.538382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 562/1122] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:50:56 2022] GET /identity/v3/projects/46833a9f550143588c1c1675edf4615e => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:56.544151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed660ee-fc6f-44dd-838c-85ad4904f475 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:56.631347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:56.641087 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed660ee-fc6f-44dd-838c-85ad4904f475 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:56.673221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d78dfd-569f-4d70-92bb-cf9044500338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:56.674087 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d78dfd-569f-4d70-92bb-cf9044500338 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.674364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d78dfd-569f-4d70-92bb-cf9044500338 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.674635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d78dfd-569f-4d70-92bb-cf9044500338 None admin] PATH_INFO: `/v3/projects/46833a9f550143588c1c1675edf4615e/users/d0f4a24f40524bc9b4acd1e7acd7d218/roles/8a0cfaebc67a497a98459da3f1a82ef9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.676792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d78dfd-569f-4d70-92bb-cf9044500338 None admin] RBAC: Authorizing `identity:create_grant(project_id=46833a9f550143588c1c1675edf4615e, user_id=d0f4a24f40524bc9b4acd1e7acd7d218, role_id=8a0cfaebc67a497a98459da3f1a82ef9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:56.678197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d78dfd-569f-4d70-92bb-cf9044500338 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:56.690700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 561/1123] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:56 2022] PUT /identity/v3/projects/46833a9f550143588c1c1675edf4615e/users/d0f4a24f40524bc9b4acd1e7acd7d218/roles/8a0cfaebc67a497a98459da3f1a82ef9 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:58.651115 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278fd78-02b7-4dfd-81d9-8b7f3a4d3d46 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:58.651727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278fd78-02b7-4dfd-81d9-8b7f3a4d3d46 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:58.651727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278fd78-02b7-4dfd-81d9-8b7f3a4d3d46 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:58.653031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 563/1124] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:58.658184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc43a3-a174-4f97-9edf-e56d8d05b456 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:58.658571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc43a3-a174-4f97-9edf-e56d8d05b456 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:58.658738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc43a3-a174-4f97-9edf-e56d8d05b456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:58.675397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-11dc43a3-a174-4f97-9edf-e56d8d05b456 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:58.676602 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-11dc43a3-a174-4f97-9edf-e56d8d05b456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:58.740566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 562/1125] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:58.764100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278fd78-02b7-4dfd-81d9-8b7f3a4d3d46 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:58.853351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:58.863099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278fd78-02b7-4dfd-81d9-8b7f3a4d3d46 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:58.895810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67decc5-4974-4e36-9a9f-1daafe7f1b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:58.896654 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67decc5-4974-4e36-9a9f-1daafe7f1b8a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:58.896937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67decc5-4974-4e36-9a9f-1daafe7f1b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:58.897212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67decc5-4974-4e36-9a9f-1daafe7f1b8a None admin] PATH_INFO: `/v3/roles/8a0cfaebc67a497a98459da3f1a82ef9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:58.898668 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67decc5-4974-4e36-9a9f-1daafe7f1b8a None admin] RBAC: Authorizing `identity:get_role(role_id=8a0cfaebc67a497a98459da3f1a82ef9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:58.900051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67decc5-4974-4e36-9a9f-1daafe7f1b8a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:58.901443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 564/1126] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:50:58 2022] GET /identity/v3/roles/8a0cfaebc67a497a98459da3f1a82ef9 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:58.907704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc43a3-a174-4f97-9edf-e56d8d05b456 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:58.984891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:59.005022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc43a3-a174-4f97-9edf-e56d8d05b456 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:59.038154 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e7b80-db5c-4798-a034-8d92c6235f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:59.039005 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e7b80-db5c-4798-a034-8d92c6235f7d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:59.039285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e7b80-db5c-4798-a034-8d92c6235f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:59.039559 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e7b80-db5c-4798-a034-8d92c6235f7d None admin] PATH_INFO: `/v3/users/d0f4a24f40524bc9b4acd1e7acd7d218` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:59.040759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e7b80-db5c-4798-a034-8d92c6235f7d None admin] RBAC: Authorizing `identity:get_user(user_id=d0f4a24f40524bc9b4acd1e7acd7d218)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:59.042198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e7b80-db5c-4798-a034-8d92c6235f7d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:59.043826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 563/1127] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:50:58 2022] GET /identity/v3/users/d0f4a24f40524bc9b4acd1e7acd7d218 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:59.049382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67decc5-4974-4e36-9a9f-1daafe7f1b8a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:59.145484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67decc5-4974-4e36-9a9f-1daafe7f1b8a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:59.178013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197315e-a035-48fb-9542-8b9eaacc9c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:59.178597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197315e-a035-48fb-9542-8b9eaacc9c2e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:59.178880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197315e-a035-48fb-9542-8b9eaacc9c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:59.179157 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197315e-a035-48fb-9542-8b9eaacc9c2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:59.206150 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:59.244658 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197315e-a035-48fb-9542-8b9eaacc9c2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:59.246092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197315e-a035-48fb-9542-8b9eaacc9c2e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:59.338861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 565/1128] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:59.344656 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e7b80-db5c-4798-a034-8d92c6235f7d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:59.439807 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e7b80-db5c-4798-a034-8d92c6235f7d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:59.471158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5be3d2-b6a2-42e6-9680-913bdf39a70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:59.472020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5be3d2-b6a2-42e6-9680-913bdf39a70d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:59.472303 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5be3d2-b6a2-42e6-9680-913bdf39a70d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:59.472577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5be3d2-b6a2-42e6-9680-913bdf39a70d None admin] PATH_INFO: `/v3/projects/46833a9f550143588c1c1675edf4615e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:59.473730 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5be3d2-b6a2-42e6-9680-913bdf39a70d None admin] RBAC: Authorizing `identity:get_project(project_id=46833a9f550143588c1c1675edf4615e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:59.475141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5be3d2-b6a2-42e6-9680-913bdf39a70d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:59.476813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 564/1129] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:50:59 2022] GET /identity/v3/projects/46833a9f550143588c1c1675edf4615e => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:59.483077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197315e-a035-48fb-9542-8b9eaacc9c2e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:59.538280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:59.577199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197315e-a035-48fb-9542-8b9eaacc9c2e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:59.609525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea069f1f-355e-471c-bd92-fac24dfa572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:59.610153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea069f1f-355e-471c-bd92-fac24dfa572a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:59.610433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea069f1f-355e-471c-bd92-fac24dfa572a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:59.610709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea069f1f-355e-471c-bd92-fac24dfa572a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:59.611522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea069f1f-355e-471c-bd92-fac24dfa572a None admin] RBAC: Adding query filter params (role.id=8a0cfaebc67a497a98459da3f1a82ef9, scope.project.id=46833a9f550143588c1c1675edf4615e, user.id=d0f4a24f40524bc9b4acd1e7acd7d218) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:59.611860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea069f1f-355e-471c-bd92-fac24dfa572a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:59.613260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea069f1f-355e-471c-bd92-fac24dfa572a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:59.618815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 566/1130] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:59 2022] GET /identity/v3/role_assignments?user.id=d0f4a24f40524bc9b4acd1e7acd7d218&scope.project.id=46833a9f550143588c1c1675edf4615e&role.id=8a0cfaebc67a497a98459da3f1a82ef9 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:01.639127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3d5c0-bd55-467c-bcaa-5140de737004 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:01.639702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3d5c0-bd55-467c-bcaa-5140de737004 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:01.639702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3d5c0-bd55-467c-bcaa-5140de737004 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:01.641072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 565/1131] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:01.646304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b96eb0-c5f2-439b-b6e3-14a2343f3577 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:01.646722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b96eb0-c5f2-439b-b6e3-14a2343f3577 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:01.646863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b96eb0-c5f2-439b-b6e3-14a2343f3577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:01.663866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-66b96eb0-c5f2-439b-b6e3-14a2343f3577 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:01.665048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-66b96eb0-c5f2-439b-b6e3-14a2343f3577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:01.729303 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 567/1132] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:01.752257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3d5c0-bd55-467c-bcaa-5140de737004 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:01.770230 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:01.850541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3d5c0-bd55-467c-bcaa-5140de737004 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:01.882913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c5d23-b16c-4389-9caa-38455e37c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:01.883706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c5d23-b16c-4389-9caa-38455e37c579 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:01.883989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c5d23-b16c-4389-9caa-38455e37c579 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:01.884260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c5d23-b16c-4389-9caa-38455e37c579 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:01.885410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c5d23-b16c-4389-9caa-38455e37c579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:01.886855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c5d23-b16c-4389-9caa-38455e37c579 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:01.888996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 566/1133] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:51:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:01.894785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b96eb0-c5f2-439b-b6e3-14a2343f3577 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:01.956637 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:02.000493 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b96eb0-c5f2-439b-b6e3-14a2343f3577 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:02.033032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9c690-bd44-4937-afc0-4f6ced1d8c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:02.033680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9c690-bd44-4937-afc0-4f6ced1d8c1b None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:02.033963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9c690-bd44-4937-afc0-4f6ced1d8c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:02.034238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9c690-bd44-4937-afc0-4f6ced1d8c1b None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:02.034953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9c690-bd44-4937-afc0-4f6ced1d8c1b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:02.036400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9c690-bd44-4937-afc0-4f6ced1d8c1b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:02.060840 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 568/1134] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:51:01 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:51:04.076868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244e5f0-be23-4830-9e57-37f51e39c5c5 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.077435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244e5f0-be23-4830-9e57-37f51e39c5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.077435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244e5f0-be23-4830-9e57-37f51e39c5c5 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.078792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 567/1135] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:51:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:04.083959 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545afd0e-3f81-4bcc-a23d-77d70b7c1690 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.084338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545afd0e-3f81-4bcc-a23d-77d70b7c1690 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.084509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545afd0e-3f81-4bcc-a23d-77d70b7c1690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.101597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-545afd0e-3f81-4bcc-a23d-77d70b7c1690 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:04.102762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-545afd0e-3f81-4bcc-a23d-77d70b7c1690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:04.166359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 569/1136] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:04.190501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244e5f0-be23-4830-9e57-37f51e39c5c5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:04.273545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:04.289831 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244e5f0-be23-4830-9e57-37f51e39c5c5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:04.322163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c33cf2-769c-41bc-8fb1-fdd46523bb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:04.322938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c33cf2-769c-41bc-8fb1-fdd46523bb35 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.323227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c33cf2-769c-41bc-8fb1-fdd46523bb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.323500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c33cf2-769c-41bc-8fb1-fdd46523bb35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.324690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c33cf2-769c-41bc-8fb1-fdd46523bb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:04.326709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c33cf2-769c-41bc-8fb1-fdd46523bb35 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:04.328388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 568/1137] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:51:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:04.334013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545afd0e-3f81-4bcc-a23d-77d70b7c1690 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:04.367005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:04.430277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545afd0e-3f81-4bcc-a23d-77d70b7c1690 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:04.463233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfa99c-6f29-4a7a-aeeb-a4016583cea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:04.463920 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfa99c-6f29-4a7a-aeeb-a4016583cea8 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.464206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfa99c-6f29-4a7a-aeeb-a4016583cea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.464484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfa99c-6f29-4a7a-aeeb-a4016583cea8 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.465199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfa99c-6f29-4a7a-aeeb-a4016583cea8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:04.466648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfa99c-6f29-4a7a-aeeb-a4016583cea8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:04.485260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 570/1138] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:51:04 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:51:06.521087 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050daaf-d3aa-4852-ab22-ad3cdd0c3724 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:06.521589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050daaf-d3aa-4852-ab22-ad3cdd0c3724 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:06.521704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050daaf-d3aa-4852-ab22-ad3cdd0c3724 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:06.522993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 569/1139] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:06.528100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd15f36-e175-4b5b-b36d-5384e09022c5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:06.528508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd15f36-e175-4b5b-b36d-5384e09022c5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:06.528666 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd15f36-e175-4b5b-b36d-5384e09022c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:06.544954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4fd15f36-e175-4b5b-b36d-5384e09022c5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:06.546130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4fd15f36-e175-4b5b-b36d-5384e09022c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:06.610270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 571/1140] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:51:06.633556 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050daaf-d3aa-4852-ab22-ad3cdd0c3724 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:06.730801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050daaf-d3aa-4852-ab22-ad3cdd0c3724 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:06.763474 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea924aa-adb6-4445-ab35-b740fd4fe350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:06.764264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea924aa-adb6-4445-ab35-b740fd4fe350 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:06.764545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea924aa-adb6-4445-ab35-b740fd4fe350 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:06.764817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea924aa-adb6-4445-ab35-b740fd4fe350 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:06.774597 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea924aa-adb6-4445-ab35-b740fd4fe350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1ea924aa-adb6-4445-ab35-b740fd4fe350 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:06.782594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:06.783815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:51:06.784947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 570/1141] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:06.789446 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd15f36-e175-4b5b-b36d-5384e09022c5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:06.812905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:06.886309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd15f36-e175-4b5b-b36d-5384e09022c5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:06.919206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e5b0b-1fb8-4e34-9ccb-92324acb7985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:06.919919 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e5b0b-1fb8-4e34-9ccb-92324acb7985 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:06.920207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e5b0b-1fb8-4e34-9ccb-92324acb7985 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:06.920488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e5b0b-1fb8-4e34-9ccb-92324acb7985 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:06.921231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e5b0b-1fb8-4e34-9ccb-92324acb7985 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:06.921543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e5b0b-1fb8-4e34-9ccb-92324acb7985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:06.922908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e5b0b-1fb8-4e34-9ccb-92324acb7985 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:06.930759 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 572/1142] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:06.936423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea924aa-adb6-4445-ab35-b740fd4fe350 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:06.951810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:07.034683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea924aa-adb6-4445-ab35-b740fd4fe350 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:07.066605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af1fa5-3e9f-4392-89be-786fe54190f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:07.067449 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af1fa5-3e9f-4392-89be-786fe54190f6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:07.067757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af1fa5-3e9f-4392-89be-786fe54190f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:07.068035 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af1fa5-3e9f-4392-89be-786fe54190f6 None admin] PATH_INFO: `/v3/users/9e96fc0bfe6f4c89b42ec453f9a1da18` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:07.081198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af1fa5-3e9f-4392-89be-786fe54190f6 None admin] RBAC: Authorizing `identity:get_user(user_id=9e96fc0bfe6f4c89b42ec453f9a1da18)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:07.082612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af1fa5-3e9f-4392-89be-786fe54190f6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:07.084252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 571/1143] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:06 2022] GET /identity/v3/users/9e96fc0bfe6f4c89b42ec453f9a1da18 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:07.090050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e5b0b-1fb8-4e34-9ccb-92324acb7985 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:07.172516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:07.188085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e5b0b-1fb8-4e34-9ccb-92324acb7985 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:07.220910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84311d1d-5e30-46c5-a704-19dc72c743c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:07.221551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84311d1d-5e30-46c5-a704-19dc72c743c2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:07.221836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84311d1d-5e30-46c5-a704-19dc72c743c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:07.222119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84311d1d-5e30-46c5-a704-19dc72c743c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:07.287040 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84311d1d-5e30-46c5-a704-19dc72c743c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:07.288454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84311d1d-5e30-46c5-a704-19dc72c743c2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:07.367228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:07.383920 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 573/1144] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:07.390343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af1fa5-3e9f-4392-89be-786fe54190f6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:07.456968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:07.486058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af1fa5-3e9f-4392-89be-786fe54190f6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:07.519396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b4400-0fce-4bb1-babf-5653da93541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:07.520271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b4400-0fce-4bb1-babf-5653da93541c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:07.520553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b4400-0fce-4bb1-babf-5653da93541c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:07.520828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b4400-0fce-4bb1-babf-5653da93541c None admin] PATH_INFO: `/v3/projects/68b0f5a94a784f08a4491b29d613364e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:07.521999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b4400-0fce-4bb1-babf-5653da93541c None admin] RBAC: Authorizing `identity:get_project(project_id=68b0f5a94a784f08a4491b29d613364e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:07.523422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b4400-0fce-4bb1-babf-5653da93541c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:07.525124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 572/1145] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:07 2022] GET /identity/v3/projects/68b0f5a94a784f08a4491b29d613364e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:07.530927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84311d1d-5e30-46c5-a704-19dc72c743c2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:07.628024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84311d1d-5e30-46c5-a704-19dc72c743c2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:07.660934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b865fe-76d4-4d70-a8d0-3ab446233ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:07.661577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b865fe-76d4-4d70-a8d0-3ab446233ffc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:07.661862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b865fe-76d4-4d70-a8d0-3ab446233ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:07.662140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b865fe-76d4-4d70-a8d0-3ab446233ffc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:07.662962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b865fe-76d4-4d70-a8d0-3ab446233ffc None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=68b0f5a94a784f08a4491b29d613364e, user.id=9e96fc0bfe6f4c89b42ec453f9a1da18) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:07.663276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b865fe-76d4-4d70-a8d0-3ab446233ffc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:07.664733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b865fe-76d4-4d70-a8d0-3ab446233ffc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:07.670503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 574/1146] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:07 2022] GET /identity/v3/role_assignments?user.id=9e96fc0bfe6f4c89b42ec453f9a1da18&scope.project.id=68b0f5a94a784f08a4491b29d613364e&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:09.658860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da541-c4fd-4d1d-a885-f11fba2d00ca None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:09.659383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da541-c4fd-4d1d-a885-f11fba2d00ca None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:09.659477 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da541-c4fd-4d1d-a885-f11fba2d00ca None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:09.660858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 573/1147] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:09.666141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85089e4a-63f3-4d1c-90da-757d094713fd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:09.666565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85089e4a-63f3-4d1c-90da-757d094713fd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:09.666705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85089e4a-63f3-4d1c-90da-757d094713fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:09.683687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-85089e4a-63f3-4d1c-90da-757d094713fd None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:09.684877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-85089e4a-63f3-4d1c-90da-757d094713fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:09.726956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:09.750076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 575/1148] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:09.772386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da541-c4fd-4d1d-a885-f11fba2d00ca None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:09.870879 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da541-c4fd-4d1d-a885-f11fba2d00ca None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:09.904573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e574ee0-2484-4bbc-b31c-fccf86b53492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:09.905358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e574ee0-2484-4bbc-b31c-fccf86b53492 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:09.905643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e574ee0-2484-4bbc-b31c-fccf86b53492 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:09.905921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e574ee0-2484-4bbc-b31c-fccf86b53492 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:09.912609 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:09.916723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e574ee0-2484-4bbc-b31c-fccf86b53492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6e574ee0-2484-4bbc-b31c-fccf86b53492 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:09.924480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:09.925591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:51:09.926646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 574/1149] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:09.931683 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85089e4a-63f3-4d1c-90da-757d094713fd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:10.032629 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85089e4a-63f3-4d1c-90da-757d094713fd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:10.064605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecc0ca-c4d2-4c1a-a750-6c4bf25a9b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:10.065258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecc0ca-c4d2-4c1a-a750-6c4bf25a9b4d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:10.065541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecc0ca-c4d2-4c1a-a750-6c4bf25a9b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:10.065817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecc0ca-c4d2-4c1a-a750-6c4bf25a9b4d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:10.066564 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecc0ca-c4d2-4c1a-a750-6c4bf25a9b4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:10.066874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecc0ca-c4d2-4c1a-a750-6c4bf25a9b4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:10.068259 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecc0ca-c4d2-4c1a-a750-6c4bf25a9b4d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:10.076581 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 576/1150] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:10.082412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e574ee0-2484-4bbc-b31c-fccf86b53492 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:10.178726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e574ee0-2484-4bbc-b31c-fccf86b53492 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:10.210892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe9808-2deb-40b7-bcce-97b3dd4a992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:10.211780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe9808-2deb-40b7-bcce-97b3dd4a992c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:10.212070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe9808-2deb-40b7-bcce-97b3dd4a992c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:10.212350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe9808-2deb-40b7-bcce-97b3dd4a992c None admin] PATH_INFO: `/v3/users/9e96fc0bfe6f4c89b42ec453f9a1da18` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:10.213544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe9808-2deb-40b7-bcce-97b3dd4a992c None admin] RBAC: Authorizing `identity:get_user(user_id=9e96fc0bfe6f4c89b42ec453f9a1da18)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:10.214970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe9808-2deb-40b7-bcce-97b3dd4a992c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:10.217218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 575/1151] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:10 2022] GET /identity/v3/users/9e96fc0bfe6f4c89b42ec453f9a1da18 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:10.222949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecc0ca-c4d2-4c1a-a750-6c4bf25a9b4d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:10.266635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:10.321178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecc0ca-c4d2-4c1a-a750-6c4bf25a9b4d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:10.352938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41480639-edd2-4be1-ae1d-4a528bc5e0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:10.353780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41480639-edd2-4be1-ae1d-4a528bc5e0aa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:10.354061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41480639-edd2-4be1-ae1d-4a528bc5e0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:10.354333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41480639-edd2-4be1-ae1d-4a528bc5e0aa None admin] PATH_INFO: `/v3/projects/68b0f5a94a784f08a4491b29d613364e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:10.355476 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41480639-edd2-4be1-ae1d-4a528bc5e0aa None admin] RBAC: Authorizing `identity:get_project(project_id=68b0f5a94a784f08a4491b29d613364e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:10.356955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41480639-edd2-4be1-ae1d-4a528bc5e0aa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:10.358606 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 577/1152] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:10 2022] GET /identity/v3/projects/68b0f5a94a784f08a4491b29d613364e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:10.364673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe9808-2deb-40b7-bcce-97b3dd4a992c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:10.426795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:10.461377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe9808-2deb-40b7-bcce-97b3dd4a992c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:10.492806 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068e995-870b-4109-b9c1-715519b8c340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:10.493680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068e995-870b-4109-b9c1-715519b8c340 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:10.493961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068e995-870b-4109-b9c1-715519b8c340 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:10.494236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068e995-870b-4109-b9c1-715519b8c340 None admin] PATH_INFO: `/v3/projects/68b0f5a94a784f08a4491b29d613364e/users/9e96fc0bfe6f4c89b42ec453f9a1da18/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:10.496192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068e995-870b-4109-b9c1-715519b8c340 None admin] RBAC: Authorizing `identity:create_grant(project_id=68b0f5a94a784f08a4491b29d613364e, user_id=9e96fc0bfe6f4c89b42ec453f9a1da18, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:10.498127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068e995-870b-4109-b9c1-715519b8c340 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:10.511416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 576/1153] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:51:10 2022] PUT /identity/v3/projects/68b0f5a94a784f08a4491b29d613364e/users/9e96fc0bfe6f4c89b42ec453f9a1da18/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:51:12.476069 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cee21-cd67-4b1a-b1ef-5487ca52f3d2 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:12.476589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cee21-cd67-4b1a-b1ef-5487ca52f3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:12.476589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cee21-cd67-4b1a-b1ef-5487ca52f3d2 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:12.477912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 578/1154] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:12.483572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae05e6-bb98-4576-bbcc-38d905e1236a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:12.483973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae05e6-bb98-4576-bbcc-38d905e1236a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:12.484162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae05e6-bb98-4576-bbcc-38d905e1236a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:12.500757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6dae05e6-bb98-4576-bbcc-38d905e1236a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:12.501922 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6dae05e6-bb98-4576-bbcc-38d905e1236a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:12.565001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 577/1155] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:12.588133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cee21-cd67-4b1a-b1ef-5487ca52f3d2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:12.686741 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cee21-cd67-4b1a-b1ef-5487ca52f3d2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:12.695416 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:12.719315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbfd30-1a4a-4911-a701-b492d5a1478e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:12.720121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbfd30-1a4a-4911-a701-b492d5a1478e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:12.720403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbfd30-1a4a-4911-a701-b492d5a1478e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:12.720676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbfd30-1a4a-4911-a701-b492d5a1478e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:12.730285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cbfd30-1a4a-4911-a701-b492d5a1478e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-57cbfd30-1a4a-4911-a701-b492d5a1478e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:12.738618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:12.739757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:12.740832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:12.740832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:12.740832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:12.740832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:12.740832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:12.740832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:12.740832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:12.740832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:12.740832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:12.740832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:12.740832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:12.740832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:51:12.743817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 579/1156] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:12.745578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae05e6-bb98-4576-bbcc-38d905e1236a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:12.799231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:12.840962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae05e6-bb98-4576-bbcc-38d905e1236a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:12.874578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b521-0e90-4266-b5dc-fbeb21362190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:12.875237 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b521-0e90-4266-b5dc-fbeb21362190 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:12.875518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b521-0e90-4266-b5dc-fbeb21362190 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:12.875820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b521-0e90-4266-b5dc-fbeb21362190 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:12.876553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790b521-0e90-4266-b5dc-fbeb21362190 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:12.876858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790b521-0e90-4266-b5dc-fbeb21362190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:12.878205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790b521-0e90-4266-b5dc-fbeb21362190 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:12.885998 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 578/1157] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:12.892274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbfd30-1a4a-4911-a701-b492d5a1478e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:12.989771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbfd30-1a4a-4911-a701-b492d5a1478e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:13.021850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a5919-3194-47cc-978a-0184ab1b802f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:13.022704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a5919-3194-47cc-978a-0184ab1b802f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:13.022986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a5919-3194-47cc-978a-0184ab1b802f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:13.023263 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a5919-3194-47cc-978a-0184ab1b802f None admin] PATH_INFO: `/v3/users/9e96fc0bfe6f4c89b42ec453f9a1da18` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:13.024458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a5919-3194-47cc-978a-0184ab1b802f None admin] RBAC: Authorizing `identity:get_user(user_id=9e96fc0bfe6f4c89b42ec453f9a1da18)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:13.025870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a5919-3194-47cc-978a-0184ab1b802f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:13.027453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 580/1158] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:12 2022] GET /identity/v3/users/9e96fc0bfe6f4c89b42ec453f9a1da18 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:13.033620 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b521-0e90-4266-b5dc-fbeb21362190 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:13.130415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b521-0e90-4266-b5dc-fbeb21362190 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:13.146911 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:13.163814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7f1df-3e69-45eb-af97-a80742a02a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:13.164461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7f1df-3e69-45eb-af97-a80742a02a1f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:13.164744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7f1df-3e69-45eb-af97-a80742a02a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:13.165029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7f1df-3e69-45eb-af97-a80742a02a1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:13.228783 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7f1df-3e69-45eb-af97-a80742a02a1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:13.230217 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7f1df-3e69-45eb-af97-a80742a02a1f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:13.323350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 579/1159] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:13.329824 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a5919-3194-47cc-978a-0184ab1b802f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:13.364360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:13.425976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a5919-3194-47cc-978a-0184ab1b802f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:13.458432 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e011f-9301-4f6c-b8c4-bda2cb70d9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:13.459269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e011f-9301-4f6c-b8c4-bda2cb70d9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:13.459551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e011f-9301-4f6c-b8c4-bda2cb70d9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:13.459850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e011f-9301-4f6c-b8c4-bda2cb70d9e5 None admin] PATH_INFO: `/v3/projects/68b0f5a94a784f08a4491b29d613364e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:13.461012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e011f-9301-4f6c-b8c4-bda2cb70d9e5 None admin] RBAC: Authorizing `identity:get_project(project_id=68b0f5a94a784f08a4491b29d613364e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:13.462425 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e011f-9301-4f6c-b8c4-bda2cb70d9e5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:13.464105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 581/1160] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:13 2022] GET /identity/v3/projects/68b0f5a94a784f08a4491b29d613364e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:13.470103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7f1df-3e69-45eb-af97-a80742a02a1f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:13.488004 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:13.567000 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7f1df-3e69-45eb-af97-a80742a02a1f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:13.599858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13832d8d-263a-4162-b11c-84acaa389a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:13.600452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13832d8d-263a-4162-b11c-84acaa389a83 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:13.600738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13832d8d-263a-4162-b11c-84acaa389a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:13.601018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13832d8d-263a-4162-b11c-84acaa389a83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:13.601838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13832d8d-263a-4162-b11c-84acaa389a83 None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=68b0f5a94a784f08a4491b29d613364e, user.id=9e96fc0bfe6f4c89b42ec453f9a1da18) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:13.602153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13832d8d-263a-4162-b11c-84acaa389a83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:13.603603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13832d8d-263a-4162-b11c-84acaa389a83 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:13.610098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 580/1161] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:13 2022] GET /identity/v3/role_assignments?user.id=9e96fc0bfe6f4c89b42ec453f9a1da18&scope.project.id=68b0f5a94a784f08a4491b29d613364e&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:15.616388 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d271ed-c851-4cee-accd-4e175c54702d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:15.616935 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d271ed-c851-4cee-accd-4e175c54702d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:15.616935 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d271ed-c851-4cee-accd-4e175c54702d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:15.618306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 582/1162] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:15.623430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea8d35-c174-4d00-9a5c-8ca84e3209fa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:15.623874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea8d35-c174-4d00-9a5c-8ca84e3209fa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:15.624018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea8d35-c174-4d00-9a5c-8ca84e3209fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:15.640387 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8dea8d35-c174-4d00-9a5c-8ca84e3209fa None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:15.641576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8dea8d35-c174-4d00-9a5c-8ca84e3209fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:15.705621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 581/1163] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:15.728272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d271ed-c851-4cee-accd-4e175c54702d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:15.826036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d271ed-c851-4cee-accd-4e175c54702d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:15.828313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:15.859827 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b876a-b832-4e9a-99eb-ef6e2de8fb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:15.860460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b876a-b832-4e9a-99eb-ef6e2de8fb70 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:15.860742 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b876a-b832-4e9a-99eb-ef6e2de8fb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:15.861018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b876a-b832-4e9a-99eb-ef6e2de8fb70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:15.924919 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b876a-b832-4e9a-99eb-ef6e2de8fb70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:15.926291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b876a-b832-4e9a-99eb-ef6e2de8fb70 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:16.021006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 583/1164] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:16.026938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea8d35-c174-4d00-9a5c-8ca84e3209fa None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:16.040496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:16.123057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea8d35-c174-4d00-9a5c-8ca84e3209fa None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:16.156202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0985c3c0-0d00-46fc-99dd-e7d2ad690129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:16.156989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985c3c0-0d00-46fc-99dd-e7d2ad690129 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:16.157269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985c3c0-0d00-46fc-99dd-e7d2ad690129 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:16.157539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985c3c0-0d00-46fc-99dd-e7d2ad690129 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:16.164011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0985c3c0-0d00-46fc-99dd-e7d2ad690129 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:16.165438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0985c3c0-0d00-46fc-99dd-e7d2ad690129 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0985c3c0-0d00-46fc-99dd-e7d2ad690129 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:16.179205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:16.180546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:51:16.181424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 582/1165] 10.222.0.65 () {56 vars in 1117 bytes} [Thu Jun 23 10:51:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:16.186145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b876a-b832-4e9a-99eb-ef6e2de8fb70 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:16.189187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:16.282233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b876a-b832-4e9a-99eb-ef6e2de8fb70 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:16.314596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156df22e-e652-4b5d-91ae-71252106eed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:16.315248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156df22e-e652-4b5d-91ae-71252106eed7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:16.315533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156df22e-e652-4b5d-91ae-71252106eed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:16.315838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156df22e-e652-4b5d-91ae-71252106eed7 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:16.316552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156df22e-e652-4b5d-91ae-71252106eed7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:16.316861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156df22e-e652-4b5d-91ae-71252106eed7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:16.318217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156df22e-e652-4b5d-91ae-71252106eed7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:16.325093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 584/1166] 10.222.0.65 () {56 vars in 1115 bytes} [Thu Jun 23 10:51:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:18.289983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa160fd-0bf2-4a35-8df9-a32d06f3570b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:18.290498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa160fd-0bf2-4a35-8df9-a32d06f3570b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:18.290613 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa160fd-0bf2-4a35-8df9-a32d06f3570b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:18.291931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 583/1167] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:18.297605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2526a7-c7e8-45d8-88cd-8ed1805b662c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:18.298004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2526a7-c7e8-45d8-88cd-8ed1805b662c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:18.298150 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2526a7-c7e8-45d8-88cd-8ed1805b662c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:18.314946 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fd2526a7-c7e8-45d8-88cd-8ed1805b662c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:18.316141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fd2526a7-c7e8-45d8-88cd-8ed1805b662c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:18.375191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:18.380705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 585/1168] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:18.403062 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa160fd-0bf2-4a35-8df9-a32d06f3570b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:18.472419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:18.503723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa160fd-0bf2-4a35-8df9-a32d06f3570b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:18.535611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e9292-938b-4a78-94c8-7b0754f245f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:18.536287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e9292-938b-4a78-94c8-7b0754f245f0 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:18.536571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e9292-938b-4a78-94c8-7b0754f245f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:18.536846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e9292-938b-4a78-94c8-7b0754f245f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:18.537579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e9292-938b-4a78-94c8-7b0754f245f0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:18.538934 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e9292-938b-4a78-94c8-7b0754f245f0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:18.553394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 584/1169] 10.222.0.65 () {60 vars in 1138 bytes} [Thu Jun 23 10:51:18 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:51:20.557444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3e92b-d864-4998-a410-4e4d58533149 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:20.557980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3e92b-d864-4998-a410-4e4d58533149 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:20.557980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3e92b-d864-4998-a410-4e4d58533149 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:20.559343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 586/1170] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:20.564944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e9242-c078-4ae2-8508-062f1ce0c3a3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:20.565360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e9242-c078-4ae2-8508-062f1ce0c3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:20.565512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e9242-c078-4ae2-8508-062f1ce0c3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:20.581931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f02e9242-c078-4ae2-8508-062f1ce0c3a3 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:20.583114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f02e9242-c078-4ae2-8508-062f1ce0c3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:20.647186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 585/1171] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:20.670740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3e92b-d864-4998-a410-4e4d58533149 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:20.768540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3e92b-d864-4998-a410-4e4d58533149 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:20.801344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7113f3-0b14-4cd7-b00b-9e8914008c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:20.802182 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7113f3-0b14-4cd7-b00b-9e8914008c63 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:20.802462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7113f3-0b14-4cd7-b00b-9e8914008c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:20.802736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7113f3-0b14-4cd7-b00b-9e8914008c63 None admin] PATH_INFO: `/v3/domains/503e2e4e466e49eba41a9515f1184d0c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:20.812827 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7113f3-0b14-4cd7-b00b-9e8914008c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=503e2e4e466e49eba41a9515f1184d0c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:20.814186 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7113f3-0b14-4cd7-b00b-9e8914008c63 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:20.815784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 587/1172] 10.222.0.65 () {56 vars in 1183 bytes} [Thu Jun 23 10:51:20 2022] GET /identity/v3/domains/503e2e4e466e49eba41a9515f1184d0c => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:20.821457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e9242-c078-4ae2-8508-062f1ce0c3a3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:20.880735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:20.918901 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e9242-c078-4ae2-8508-062f1ce0c3a3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:20.951510 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdf15a-fb1b-4e80-b0de-66b32d7a56fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:20.952182 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdf15a-fb1b-4e80-b0de-66b32d7a56fe None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:20.952462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdf15a-fb1b-4e80-b0de-66b32d7a56fe None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:20.952737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdf15a-fb1b-4e80-b0de-66b32d7a56fe None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:20.953437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdf15a-fb1b-4e80-b0de-66b32d7a56fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:20.954860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdf15a-fb1b-4e80-b0de-66b32d7a56fe None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:20.978498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 586/1173] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:51:20 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:51:23.009880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864ab4d-2fe2-41cc-8f98-14bcdf12ccea None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:23.010425 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864ab4d-2fe2-41cc-8f98-14bcdf12ccea None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:23.010425 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864ab4d-2fe2-41cc-8f98-14bcdf12ccea None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:23.011805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 588/1174] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:23.017498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc5533-5eff-4a3f-8c4f-86d488df3909 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:23.017880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc5533-5eff-4a3f-8c4f-86d488df3909 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:23.018061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc5533-5eff-4a3f-8c4f-86d488df3909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:23.034398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-75bc5533-5eff-4a3f-8c4f-86d488df3909 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:23.035572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-75bc5533-5eff-4a3f-8c4f-86d488df3909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:23.100323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 587/1175] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:23.125704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864ab4d-2fe2-41cc-8f98-14bcdf12ccea None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:23.181974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:23.223570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864ab4d-2fe2-41cc-8f98-14bcdf12ccea None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:23.255916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9db82c-d301-48b8-b6cd-b09a686007ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:23.256753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9db82c-d301-48b8-b6cd-b09a686007ca None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:23.257031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9db82c-d301-48b8-b6cd-b09a686007ca None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:23.257299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9db82c-d301-48b8-b6cd-b09a686007ca None admin] PATH_INFO: `/v3/domains/503e2e4e466e49eba41a9515f1184d0c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:23.258444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9db82c-d301-48b8-b6cd-b09a686007ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=503e2e4e466e49eba41a9515f1184d0c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:23.259932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9db82c-d301-48b8-b6cd-b09a686007ca None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:23.261453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 589/1176] 10.222.0.65 () {56 vars in 1183 bytes} [Thu Jun 23 10:51:23 2022] GET /identity/v3/domains/503e2e4e466e49eba41a9515f1184d0c => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:23.267058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc5533-5eff-4a3f-8c4f-86d488df3909 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:23.311184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:23.364546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc5533-5eff-4a3f-8c4f-86d488df3909 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:23.397102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0aea78-3671-4f70-8c19-362b00bed3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:23.397766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0aea78-3671-4f70-8c19-362b00bed3e1 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:23.398051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0aea78-3671-4f70-8c19-362b00bed3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:23.398329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0aea78-3671-4f70-8c19-362b00bed3e1 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:23.399044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0aea78-3671-4f70-8c19-362b00bed3e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:23.400494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0aea78-3671-4f70-8c19-362b00bed3e1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:23.420251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 588/1177] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:51:23 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:51:25.452291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c719a9-b410-49e4-9ba5-61910344c85e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:25.452788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c719a9-b410-49e4-9ba5-61910344c85e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:25.452788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c719a9-b410-49e4-9ba5-61910344c85e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:25.454126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 590/1178] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:51:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:25.459217 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98107b8d-456a-45b0-b781-08012b9faa78 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:25.459594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98107b8d-456a-45b0-b781-08012b9faa78 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:25.459802 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98107b8d-456a-45b0-b781-08012b9faa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:25.475946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-98107b8d-456a-45b0-b781-08012b9faa78 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:25.477113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-98107b8d-456a-45b0-b781-08012b9faa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:25.540879 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 589/1179] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:25.564127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c719a9-b410-49e4-9ba5-61910344c85e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:25.661074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c719a9-b410-49e4-9ba5-61910344c85e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:25.686209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:25.694765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd6046-0200-40b1-a760-c6055cff815a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:25.695538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd6046-0200-40b1-a760-c6055cff815a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:25.695847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd6046-0200-40b1-a760-c6055cff815a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:25.696123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd6046-0200-40b1-a760-c6055cff815a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:25.705964 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd6046-0200-40b1-a760-c6055cff815a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-17cd6046-0200-40b1-a760-c6055cff815a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:25.713486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:25.714591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:51:25.715626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 591/1180] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:25.720641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98107b8d-456a-45b0-b781-08012b9faa78 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:25.749853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:25.817052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98107b8d-456a-45b0-b781-08012b9faa78 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:25.849448 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5870b-3ca5-48bf-a631-191d2e001e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:25.850107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5870b-3ca5-48bf-a631-191d2e001e3e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:25.850390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5870b-3ca5-48bf-a631-191d2e001e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:25.850665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5870b-3ca5-48bf-a631-191d2e001e3e None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:25.851403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5870b-3ca5-48bf-a631-191d2e001e3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:25.851739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5870b-3ca5-48bf-a631-191d2e001e3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:25.853100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5870b-3ca5-48bf-a631-191d2e001e3e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:25.861725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 590/1181] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:25.867277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd6046-0200-40b1-a760-c6055cff815a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:25.964076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd6046-0200-40b1-a760-c6055cff815a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:25.996703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c8006-bf8d-408c-9847-102fa4734056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:25.997582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c8006-bf8d-408c-9847-102fa4734056 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:25.997864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c8006-bf8d-408c-9847-102fa4734056 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:25.998140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c8006-bf8d-408c-9847-102fa4734056 None admin] PATH_INFO: `/v3/users/2211cd0d8f5f4ffeb4177b8076608948` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:26.010634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c8006-bf8d-408c-9847-102fa4734056 None admin] RBAC: Authorizing `identity:get_user(user_id=2211cd0d8f5f4ffeb4177b8076608948)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:26.012085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c8006-bf8d-408c-9847-102fa4734056 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:26.013732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 592/1182] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:25 2022] GET /identity/v3/users/2211cd0d8f5f4ffeb4177b8076608948 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:26.019284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5870b-3ca5-48bf-a631-191d2e001e3e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:26.108772 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:26.117933 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5870b-3ca5-48bf-a631-191d2e001e3e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:26.149327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c29ec4-86b3-4474-ba14-fdaa7ce1d3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:26.149966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c29ec4-86b3-4474-ba14-fdaa7ce1d3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:26.150246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c29ec4-86b3-4474-ba14-fdaa7ce1d3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:26.150518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c29ec4-86b3-4474-ba14-fdaa7ce1d3b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:26.214141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c29ec4-86b3-4474-ba14-fdaa7ce1d3b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:26.215523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c29ec4-86b3-4474-ba14-fdaa7ce1d3b3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:26.300555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:26.309972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 591/1183] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:26.315809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c8006-bf8d-408c-9847-102fa4734056 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:26.360544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:26.412232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c8006-bf8d-408c-9847-102fa4734056 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:26.444043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d80250-7c5a-4820-9d4b-0a42c9a6a5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:26.444887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d80250-7c5a-4820-9d4b-0a42c9a6a5fb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:26.445171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d80250-7c5a-4820-9d4b-0a42c9a6a5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:26.445448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d80250-7c5a-4820-9d4b-0a42c9a6a5fb None admin] PATH_INFO: `/v3/projects/1abe5d091ae34245a536126d12f36b10` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:26.446609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d80250-7c5a-4820-9d4b-0a42c9a6a5fb None admin] RBAC: Authorizing `identity:get_project(project_id=1abe5d091ae34245a536126d12f36b10)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:26.448048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d80250-7c5a-4820-9d4b-0a42c9a6a5fb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:26.449697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 593/1184] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:26 2022] GET /identity/v3/projects/1abe5d091ae34245a536126d12f36b10 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:26.455983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c29ec4-86b3-4474-ba14-fdaa7ce1d3b3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:26.550686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c29ec4-86b3-4474-ba14-fdaa7ce1d3b3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:26.583231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cb01d-f863-41de-84fb-defe81450db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:26.583901 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cb01d-f863-41de-84fb-defe81450db7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:26.584186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cb01d-f863-41de-84fb-defe81450db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:26.584459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cb01d-f863-41de-84fb-defe81450db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:26.585276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cb01d-f863-41de-84fb-defe81450db7 None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=1abe5d091ae34245a536126d12f36b10, user.id=2211cd0d8f5f4ffeb4177b8076608948) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:26.585587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cb01d-f863-41de-84fb-defe81450db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:26.587000 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cb01d-f863-41de-84fb-defe81450db7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:26.592836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 592/1185] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:26 2022] GET /identity/v3/role_assignments?user.id=2211cd0d8f5f4ffeb4177b8076608948&scope.project.id=1abe5d091ae34245a536126d12f36b10&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:28.576187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe08bb-2e18-4e24-bbd4-7dbe41e82b5d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:28.576811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe08bb-2e18-4e24-bbd4-7dbe41e82b5d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:28.576811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe08bb-2e18-4e24-bbd4-7dbe41e82b5d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:28.578102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 594/1186] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:28.583184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539b9a1-0cff-49fe-905b-517010fb52d2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:28.583630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539b9a1-0cff-49fe-905b-517010fb52d2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:28.583770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539b9a1-0cff-49fe-905b-517010fb52d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:28.600127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7539b9a1-0cff-49fe-905b-517010fb52d2 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:28.601311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7539b9a1-0cff-49fe-905b-517010fb52d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:28.650527 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:28.666721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 593/1187] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:28.690042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe08bb-2e18-4e24-bbd4-7dbe41e82b5d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:28.786898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe08bb-2e18-4e24-bbd4-7dbe41e82b5d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:28.799757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:28.821020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad93443-ca15-4cda-a104-2cdf04046551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:28.821800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93443-ca15-4cda-a104-2cdf04046551 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:28.822086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93443-ca15-4cda-a104-2cdf04046551 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:28.822365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93443-ca15-4cda-a104-2cdf04046551 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:28.832276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad93443-ca15-4cda-a104-2cdf04046551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0ad93443-ca15-4cda-a104-2cdf04046551 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:28.839805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:28.841037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:51:28.842333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 595/1188] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:28.846918 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539b9a1-0cff-49fe-905b-517010fb52d2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:28.942610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539b9a1-0cff-49fe-905b-517010fb52d2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:28.975305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0cfe2-5d89-4c04-b2a6-f998d4cb678b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:28.975988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0cfe2-5d89-4c04-b2a6-f998d4cb678b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:28.976275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0cfe2-5d89-4c04-b2a6-f998d4cb678b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:28.976555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0cfe2-5d89-4c04-b2a6-f998d4cb678b None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:28.977296 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0cfe2-5d89-4c04-b2a6-f998d4cb678b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:28.977609 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0cfe2-5d89-4c04-b2a6-f998d4cb678b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:28.978970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0cfe2-5d89-4c04-b2a6-f998d4cb678b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:28.987640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 594/1189] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:28.993126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad93443-ca15-4cda-a104-2cdf04046551 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:29.091792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad93443-ca15-4cda-a104-2cdf04046551 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:29.124650 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856e749-fbd8-445a-b029-a5916ce823a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:29.125507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856e749-fbd8-445a-b029-a5916ce823a6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:29.125790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856e749-fbd8-445a-b029-a5916ce823a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:29.126066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856e749-fbd8-445a-b029-a5916ce823a6 None admin] PATH_INFO: `/v3/users/2211cd0d8f5f4ffeb4177b8076608948` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:29.127237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856e749-fbd8-445a-b029-a5916ce823a6 None admin] RBAC: Authorizing `identity:get_user(user_id=2211cd0d8f5f4ffeb4177b8076608948)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:29.128699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856e749-fbd8-445a-b029-a5916ce823a6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:29.130303 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 596/1190] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:28 2022] GET /identity/v3/users/2211cd0d8f5f4ffeb4177b8076608948 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:29.136242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0cfe2-5d89-4c04-b2a6-f998d4cb678b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:29.184916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:29.235196 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0cfe2-5d89-4c04-b2a6-f998d4cb678b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:29.266642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22806ce-4b26-45f5-a9bb-a68d2d9b4a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:29.267488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22806ce-4b26-45f5-a9bb-a68d2d9b4a10 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:29.267803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22806ce-4b26-45f5-a9bb-a68d2d9b4a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:29.268084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22806ce-4b26-45f5-a9bb-a68d2d9b4a10 None admin] PATH_INFO: `/v3/projects/1abe5d091ae34245a536126d12f36b10` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:29.269254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22806ce-4b26-45f5-a9bb-a68d2d9b4a10 None admin] RBAC: Authorizing `identity:get_project(project_id=1abe5d091ae34245a536126d12f36b10)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:29.270682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22806ce-4b26-45f5-a9bb-a68d2d9b4a10 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:29.272388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 595/1191] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:29 2022] GET /identity/v3/projects/1abe5d091ae34245a536126d12f36b10 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:29.278076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856e749-fbd8-445a-b029-a5916ce823a6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:29.318856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:29.375108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856e749-fbd8-445a-b029-a5916ce823a6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:29.407668 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49e5e5-c3dd-49d4-ac21-a658c65f1e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:29.408550 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49e5e5-c3dd-49d4-ac21-a658c65f1e43 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:29.408832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49e5e5-c3dd-49d4-ac21-a658c65f1e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:29.409104 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49e5e5-c3dd-49d4-ac21-a658c65f1e43 None admin] PATH_INFO: `/v3/projects/1abe5d091ae34245a536126d12f36b10/users/2211cd0d8f5f4ffeb4177b8076608948/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:29.411037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49e5e5-c3dd-49d4-ac21-a658c65f1e43 None admin] RBAC: Authorizing `identity:create_grant(project_id=1abe5d091ae34245a536126d12f36b10, user_id=2211cd0d8f5f4ffeb4177b8076608948, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:29.412472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49e5e5-c3dd-49d4-ac21-a658c65f1e43 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:29.424322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 597/1192] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:51:29 2022] PUT /identity/v3/projects/1abe5d091ae34245a536126d12f36b10/users/2211cd0d8f5f4ffeb4177b8076608948/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:51:31.393978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1da4b-9f27-4cb2-a745-ec2375e0ae69 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:31.394495 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1da4b-9f27-4cb2-a745-ec2375e0ae69 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:31.394495 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1da4b-9f27-4cb2-a745-ec2375e0ae69 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:31.395899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 596/1193] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:31.400925 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7e4d3-6a5e-4a2e-b7ff-cf9aa6fd548a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:31.401320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7e4d3-6a5e-4a2e-b7ff-cf9aa6fd548a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:31.401481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7e4d3-6a5e-4a2e-b7ff-cf9aa6fd548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:31.417655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4fd7e4d3-6a5e-4a2e-b7ff-cf9aa6fd548a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:31.418814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4fd7e4d3-6a5e-4a2e-b7ff-cf9aa6fd548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:31.481710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 598/1194] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:31.505190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1da4b-9f27-4cb2-a745-ec2375e0ae69 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:31.603278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1da4b-9f27-4cb2-a745-ec2375e0ae69 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:31.619863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:31.637481 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df3515-6cc1-409a-a54f-fd7224ee0fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:31.638294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df3515-6cc1-409a-a54f-fd7224ee0fea None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:31.638576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df3515-6cc1-409a-a54f-fd7224ee0fea None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:31.638849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df3515-6cc1-409a-a54f-fd7224ee0fea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:31.648697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df3515-6cc1-409a-a54f-fd7224ee0fea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e7df3515-6cc1-409a-a54f-fd7224ee0fea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:31.656220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:31.657342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:51:31.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 597/1195] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:31.663212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7e4d3-6a5e-4a2e-b7ff-cf9aa6fd548a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:31.692453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:31.759428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7e4d3-6a5e-4a2e-b7ff-cf9aa6fd548a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:31.791018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a79d17-86e6-48cb-8abc-766804786539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:31.791688 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a79d17-86e6-48cb-8abc-766804786539 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:31.791976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a79d17-86e6-48cb-8abc-766804786539 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:31.792249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a79d17-86e6-48cb-8abc-766804786539 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:31.792980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a79d17-86e6-48cb-8abc-766804786539 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:31.793286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a79d17-86e6-48cb-8abc-766804786539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:31.795222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a79d17-86e6-48cb-8abc-766804786539 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:31.802913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 599/1196] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:31.808426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df3515-6cc1-409a-a54f-fd7224ee0fea None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:31.905107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df3515-6cc1-409a-a54f-fd7224ee0fea None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:31.937601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe75a73-dc12-4486-ae4c-3a41b96f73d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:31.938453 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe75a73-dc12-4486-ae4c-3a41b96f73d3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:31.938736 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe75a73-dc12-4486-ae4c-3a41b96f73d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:31.939011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe75a73-dc12-4486-ae4c-3a41b96f73d3 None admin] PATH_INFO: `/v3/users/2211cd0d8f5f4ffeb4177b8076608948` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:31.940212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe75a73-dc12-4486-ae4c-3a41b96f73d3 None admin] RBAC: Authorizing `identity:get_user(user_id=2211cd0d8f5f4ffeb4177b8076608948)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:31.941633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe75a73-dc12-4486-ae4c-3a41b96f73d3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:31.943224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 598/1197] 10.222.0.65 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:31 2022] GET /identity/v3/users/2211cd0d8f5f4ffeb4177b8076608948 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:31.948770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a79d17-86e6-48cb-8abc-766804786539 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:32.037394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:32.046623 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a79d17-86e6-48cb-8abc-766804786539 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:32.078138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8eaa6-58a8-4fd7-8cc4-3ebde94eb378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:32.078774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8eaa6-58a8-4fd7-8cc4-3ebde94eb378 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:32.079057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8eaa6-58a8-4fd7-8cc4-3ebde94eb378 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:32.079333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8eaa6-58a8-4fd7-8cc4-3ebde94eb378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:32.151453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8eaa6-58a8-4fd7-8cc4-3ebde94eb378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:32.152922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8eaa6-58a8-4fd7-8cc4-3ebde94eb378 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:32.238987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:32.248316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 600/1198] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:32.254140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe75a73-dc12-4486-ae4c-3a41b96f73d3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:32.298638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:32.350854 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe75a73-dc12-4486-ae4c-3a41b96f73d3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:32.383359 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930ea26-9720-4627-9161-45195712747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:32.384238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930ea26-9720-4627-9161-45195712747d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:32.384522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930ea26-9720-4627-9161-45195712747d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:32.384800 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930ea26-9720-4627-9161-45195712747d None admin] PATH_INFO: `/v3/projects/1abe5d091ae34245a536126d12f36b10` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:32.385966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930ea26-9720-4627-9161-45195712747d None admin] RBAC: Authorizing `identity:get_project(project_id=1abe5d091ae34245a536126d12f36b10)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:32.387398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930ea26-9720-4627-9161-45195712747d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:32.389635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 599/1199] 10.222.0.65 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:32 2022] GET /identity/v3/projects/1abe5d091ae34245a536126d12f36b10 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:32.395558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8eaa6-58a8-4fd7-8cc4-3ebde94eb378 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:32.490624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8eaa6-58a8-4fd7-8cc4-3ebde94eb378 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:32.523003 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd112f3-787c-47b7-ba02-95bab5f697b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:32.523638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd112f3-787c-47b7-ba02-95bab5f697b7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:32.523949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd112f3-787c-47b7-ba02-95bab5f697b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:32.524224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd112f3-787c-47b7-ba02-95bab5f697b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:32.525039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd112f3-787c-47b7-ba02-95bab5f697b7 None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=1abe5d091ae34245a536126d12f36b10, user.id=2211cd0d8f5f4ffeb4177b8076608948) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:32.525349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd112f3-787c-47b7-ba02-95bab5f697b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:32.526750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd112f3-787c-47b7-ba02-95bab5f697b7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:32.532973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 601/1200] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:32 2022] GET /identity/v3/role_assignments?user.id=2211cd0d8f5f4ffeb4177b8076608948&scope.project.id=1abe5d091ae34245a536126d12f36b10&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:42.379366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=44 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:42.530320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=72 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:26.940554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7417d52-4cb5-484f-b5b3-36fa7c5043eb None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:26.941161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7417d52-4cb5-484f-b5b3-36fa7c5043eb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:26.941161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7417d52-4cb5-484f-b5b3-36fa7c5043eb None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:26.942581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 600/1201] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:52:26.948032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb529c9-ad06-4afa-bc10-6ecf794d93d4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:26.948556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb529c9-ad06-4afa-bc10-6ecf794d93d4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:26.948652 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb529c9-ad06-4afa-bc10-6ecf794d93d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:26.969856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0bb529c9-ad06-4afa-bc10-6ecf794d93d4 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:52:26.971470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0bb529c9-ad06-4afa-bc10-6ecf794d93d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:52:27.063696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 602/1202] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:52:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:52:27.083961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7417d52-4cb5-484f-b5b3-36fa7c5043eb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:27.208254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7417d52-4cb5-484f-b5b3-36fa7c5043eb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:27.250729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58585b8d-e955-4b31-bd0d-207d6b3ea9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:27.251545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58585b8d-e955-4b31-bd0d-207d6b3ea9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:27.251867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58585b8d-e955-4b31-bd0d-207d6b3ea9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:27.252151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58585b8d-e955-4b31-bd0d-207d6b3ea9d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:27.270251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58585b8d-e955-4b31-bd0d-207d6b3ea9d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:27.271840 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58585b8d-e955-4b31-bd0d-207d6b3ea9d9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-58585b8d-e955-4b31-bd0d-207d6b3ea9d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:27.290941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:27.292235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:27.293232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:27.293232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:27.293232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:27.293232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:27.293232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 601/1203] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:27.298995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb529c9-ad06-4afa-bc10-6ecf794d93d4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:27.427029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb529c9-ad06-4afa-bc10-6ecf794d93d4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:27.460209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:27.469968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf789d5-0c98-44c5-8388-8bdf55c18ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:27.470919 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf789d5-0c98-44c5-8388-8bdf55c18ddd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:27.471383 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf789d5-0c98-44c5-8388-8bdf55c18ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:27.471679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf789d5-0c98-44c5-8388-8bdf55c18ddd None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:27.472589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf789d5-0c98-44c5-8388-8bdf55c18ddd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:27.472856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf789d5-0c98-44c5-8388-8bdf55c18ddd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:27.474346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf789d5-0c98-44c5-8388-8bdf55c18ddd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:27.492325 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 603/1204] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:27.497902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58585b8d-e955-4b31-bd0d-207d6b3ea9d9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:27.556399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:27.614235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58585b8d-e955-4b31-bd0d-207d6b3ea9d9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:27.658333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50376b-df1e-43d7-8eeb-7f25a5957d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:27.659033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50376b-df1e-43d7-8eeb-7f25a5957d3a None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:27.659312 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50376b-df1e-43d7-8eeb-7f25a5957d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:27.659612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50376b-df1e-43d7-8eeb-7f25a5957d3a None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:27.660497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50376b-df1e-43d7-8eeb-7f25a5957d3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:27.662145 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50376b-df1e-43d7-8eeb-7f25a5957d3a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:27.689226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 602/1205] 10.222.0.65 () {60 vars in 1132 bytes} [Thu Jun 23 10:52:27 2022] POST /identity/v3/users => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:52:29.971991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa65ad-be1d-45dc-aec5-9cae752bdb51 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:29.972842 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa65ad-be1d-45dc-aec5-9cae752bdb51 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:29.972842 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa65ad-be1d-45dc-aec5-9cae752bdb51 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:29.974060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 604/1206] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:52:29.979409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd864a0-0894-49ef-8fb3-cda433d5b237 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:29.979965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd864a0-0894-49ef-8fb3-cda433d5b237 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:29.980071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd864a0-0894-49ef-8fb3-cda433d5b237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:29.998380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-afd864a0-0894-49ef-8fb3-cda433d5b237 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:52:30.000012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-afd864a0-0894-49ef-8fb3-cda433d5b237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:52:30.091677 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 603/1207] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:52:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:52:30.107838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa65ad-be1d-45dc-aec5-9cae752bdb51 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:30.230912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa65ad-be1d-45dc-aec5-9cae752bdb51 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:30.277394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c3d71-32a7-4530-ae96-fad26488786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:30.278193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c3d71-32a7-4530-ae96-fad26488786b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:30.278475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c3d71-32a7-4530-ae96-fad26488786b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:30.278752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c3d71-32a7-4530-ae96-fad26488786b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:30.292269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c3d71-32a7-4530-ae96-fad26488786b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fc4c3d71-32a7-4530-ae96-fad26488786b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:52:30.301029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:30.302286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:30.303194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 605/1208] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:52:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:30.309075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd864a0-0894-49ef-8fb3-cda433d5b237 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:30.359528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:30.430489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd864a0-0894-49ef-8fb3-cda433d5b237 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:30.470496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172fa62-c856-46f6-9d0e-c79c1947f557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:30.471175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172fa62-c856-46f6-9d0e-c79c1947f557 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:30.471460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172fa62-c856-46f6-9d0e-c79c1947f557 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:30.471784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172fa62-c856-46f6-9d0e-c79c1947f557 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:30.472532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172fa62-c856-46f6-9d0e-c79c1947f557 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:30.472846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172fa62-c856-46f6-9d0e-c79c1947f557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:30.474220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172fa62-c856-46f6-9d0e-c79c1947f557 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:30.488689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 604/1209] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:52:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:52:30.494042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c3d71-32a7-4530-ae96-fad26488786b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:30.553736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:30.613094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c3d71-32a7-4530-ae96-fad26488786b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:30.652317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd240ab8-6021-4660-8754-611e21676bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:30.653150 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd240ab8-6021-4660-8754-611e21676bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:30.653430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd240ab8-6021-4660-8754-611e21676bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:30.653698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd240ab8-6021-4660-8754-611e21676bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:30.665503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd240ab8-6021-4660-8754-611e21676bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:30.667072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd240ab8-6021-4660-8754-611e21676bd5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fd240ab8-6021-4660-8754-611e21676bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:30.685767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:30.687018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:30.688007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:30.688007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:30.688007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:30.688007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:30.688007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 606/1210] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:30.692862 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172fa62-c856-46f6-9d0e-c79c1947f557 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:30.805145 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:30.808945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172fa62-c856-46f6-9d0e-c79c1947f557 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:30.847327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededb5ba-701a-4a00-b6db-2027c71a163a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:30.848034 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededb5ba-701a-4a00-b6db-2027c71a163a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:30.848299 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededb5ba-701a-4a00-b6db-2027c71a163a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:30.848578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededb5ba-701a-4a00-b6db-2027c71a163a None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:30.849309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededb5ba-701a-4a00-b6db-2027c71a163a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:30.849623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededb5ba-701a-4a00-b6db-2027c71a163a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:30.850995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededb5ba-701a-4a00-b6db-2027c71a163a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:30.868409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 605/1211] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:30.870451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd240ab8-6021-4660-8754-611e21676bd5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:30.980992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd240ab8-6021-4660-8754-611e21676bd5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:30.982737 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:31.019354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f80fa-d30a-48f6-8457-d15cfb4c7e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:31.020205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f80fa-d30a-48f6-8457-d15cfb4c7e7a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:31.020490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f80fa-d30a-48f6-8457-d15cfb4c7e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:31.020777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f80fa-d30a-48f6-8457-d15cfb4c7e7a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:31.029859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f80fa-d30a-48f6-8457-d15cfb4c7e7a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:31.031455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f80fa-d30a-48f6-8457-d15cfb4c7e7a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4e3f80fa-d30a-48f6-8457-d15cfb4c7e7a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:52:31.046665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:31.052682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:31.053672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:31.053672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:31.053672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:52:31.053672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:52:31.053672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:52:31.053672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:52:31.053672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:52:31.053672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:52:31.053672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:52:31.053672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:31.053672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 607/1212] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:31.054306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededb5ba-701a-4a00-b6db-2027c71a163a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:31.167886 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededb5ba-701a-4a00-b6db-2027c71a163a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:31.204279 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0f4ff-e98b-4667-bd7f-9a55cea0cd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:31.206568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0f4ff-e98b-4667-bd7f-9a55cea0cd7a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:31.206568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0f4ff-e98b-4667-bd7f-9a55cea0cd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:31.206568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0f4ff-e98b-4667-bd7f-9a55cea0cd7a None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:31.206568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0f4ff-e98b-4667-bd7f-9a55cea0cd7a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:31.207009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0f4ff-e98b-4667-bd7f-9a55cea0cd7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:31.208249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0f4ff-e98b-4667-bd7f-9a55cea0cd7a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:31.233578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 606/1213] 10.222.0.65 () {56 vars in 1143 bytes} [Thu Jun 23 10:52:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:52:31.235132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f80fa-d30a-48f6-8457-d15cfb4c7e7a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:31.367789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f80fa-d30a-48f6-8457-d15cfb4c7e7a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:31.412770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811c127-1632-4ee8-8ea3-897155521a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:31.413757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811c127-1632-4ee8-8ea3-897155521a8a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:31.414147 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811c127-1632-4ee8-8ea3-897155521a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:31.414685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811c127-1632-4ee8-8ea3-897155521a8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:31.466265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:31.526283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811c127-1632-4ee8-8ea3-897155521a8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:31.528860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811c127-1632-4ee8-8ea3-897155521a8a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:31.657257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 608/1214] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:52:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:52:31.664431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0f4ff-e98b-4667-bd7f-9a55cea0cd7a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:31.687298 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:31.797470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0f4ff-e98b-4667-bd7f-9a55cea0cd7a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:31.842781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a62f7-df07-48fe-a7be-fbf01deb5a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:31.843717 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a62f7-df07-48fe-a7be-fbf01deb5a14 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:31.844103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a62f7-df07-48fe-a7be-fbf01deb5a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:31.844382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a62f7-df07-48fe-a7be-fbf01deb5a14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:31.855050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a62f7-df07-48fe-a7be-fbf01deb5a14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:31.856548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a62f7-df07-48fe-a7be-fbf01deb5a14 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-971a62f7-df07-48fe-a7be-fbf01deb5a14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:31.874830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:31.881709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:31.882634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:31.882634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:31.882634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:31.882634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:31.882634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 607/1215] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:31.882916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811c127-1632-4ee8-8ea3-897155521a8a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:31.973749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:32.020258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811c127-1632-4ee8-8ea3-897155521a8a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:32.059594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7968179-64d7-4b45-9dcd-dfd7d0618c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:32.060290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7968179-64d7-4b45-9dcd-dfd7d0618c3c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:32.061144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7968179-64d7-4b45-9dcd-dfd7d0618c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:32.061425 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7968179-64d7-4b45-9dcd-dfd7d0618c3c None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:32.062145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7968179-64d7-4b45-9dcd-dfd7d0618c3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:32.062454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7968179-64d7-4b45-9dcd-dfd7d0618c3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:32.063842 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7968179-64d7-4b45-9dcd-dfd7d0618c3c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:32.080954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 609/1216] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:52:32.083738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a62f7-df07-48fe-a7be-fbf01deb5a14 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:32.170952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:32.214305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a62f7-df07-48fe-a7be-fbf01deb5a14 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:32.253900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822bb214-1976-40f6-975c-fbdbfffefa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:32.254887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bb214-1976-40f6-975c-fbdbfffefa61 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:32.255174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bb214-1976-40f6-975c-fbdbfffefa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:32.255466 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bb214-1976-40f6-975c-fbdbfffefa61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:32.263328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822bb214-1976-40f6-975c-fbdbfffefa61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:32.265046 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822bb214-1976-40f6-975c-fbdbfffefa61 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-822bb214-1976-40f6-975c-fbdbfffefa61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:52:32.280039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:32.281199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:32.287104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:52:32.287104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:52:32.287104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:52:32.287104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:52:32.287104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:52:32.287104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:52:32.287104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:52:32.287104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:32.287104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 608/1217] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:52:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:32.288996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7968179-64d7-4b45-9dcd-dfd7d0618c3c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:32.410515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7968179-64d7-4b45-9dcd-dfd7d0618c3c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:32.435361 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:32.448653 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36811a-3da0-46f5-a195-b55d93d81cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:32.449449 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36811a-3da0-46f5-a195-b55d93d81cbe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:32.449832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36811a-3da0-46f5-a195-b55d93d81cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:32.450207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36811a-3da0-46f5-a195-b55d93d81cbe None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:32.451988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36811a-3da0-46f5-a195-b55d93d81cbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:32.452585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36811a-3da0-46f5-a195-b55d93d81cbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:32.454552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36811a-3da0-46f5-a195-b55d93d81cbe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:32.469767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 610/1218] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:52:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:32.475818 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822bb214-1976-40f6-975c-fbdbfffefa61 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:32.592607 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822bb214-1976-40f6-975c-fbdbfffefa61 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:32.629662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:32.638011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5220caf-105b-48e3-b267-ca23c12acd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:32.639645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5220caf-105b-48e3-b267-ca23c12acd65 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:32.640127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5220caf-105b-48e3-b267-ca23c12acd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:32.640529 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5220caf-105b-48e3-b267-ca23c12acd65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:32.641996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5220caf-105b-48e3-b267-ca23c12acd65 None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=18fc0e7956b14a49beab025922e8ff81) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:32.642554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5220caf-105b-48e3-b267-ca23c12acd65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:32.644598 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5220caf-105b-48e3-b267-ca23c12acd65 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:32.651484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 609/1219] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:52:32 2022] GET /identity/v3/role_assignments?user.id=18fc0e7956b14a49beab025922e8ff81&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:34.847276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa3a22-ab98-4695-b430-931b0bbc4c56 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:34.847865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa3a22-ab98-4695-b430-931b0bbc4c56 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:34.847865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa3a22-ab98-4695-b430-931b0bbc4c56 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:34.849209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 611/1220] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:52:34.854510 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789e4d1-15e2-4255-ad89-acc3b5907dfd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:34.854915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789e4d1-15e2-4255-ad89-acc3b5907dfd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:34.855079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789e4d1-15e2-4255-ad89-acc3b5907dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:34.873017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2789e4d1-15e2-4255-ad89-acc3b5907dfd None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:52:34.874363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2789e4d1-15e2-4255-ad89-acc3b5907dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:52:34.957002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 610/1221] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:52:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:52:34.984099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa3a22-ab98-4695-b430-931b0bbc4c56 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:35.101832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa3a22-ab98-4695-b430-931b0bbc4c56 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:35.139616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84055f-53d2-4c06-bea3-683e9148eacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:35.140913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84055f-53d2-4c06-bea3-683e9148eacf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:35.140913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84055f-53d2-4c06-bea3-683e9148eacf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:35.142420 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84055f-53d2-4c06-bea3-683e9148eacf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:35.154036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84055f-53d2-4c06-bea3-683e9148eacf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-dd84055f-53d2-4c06-bea3-683e9148eacf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:52:35.162962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:35.163951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:35.164835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 612/1222] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:52:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:35.171146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789e4d1-15e2-4255-ad89-acc3b5907dfd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:35.279844 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789e4d1-15e2-4255-ad89-acc3b5907dfd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:35.303106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:35.315897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1312f-5aed-4b31-a60b-3d537a5459e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:35.316576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1312f-5aed-4b31-a60b-3d537a5459e0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:35.316855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1312f-5aed-4b31-a60b-3d537a5459e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:35.317161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1312f-5aed-4b31-a60b-3d537a5459e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:35.318179 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1312f-5aed-4b31-a60b-3d537a5459e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:35.318460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1312f-5aed-4b31-a60b-3d537a5459e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:35.320104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1312f-5aed-4b31-a60b-3d537a5459e0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:35.329041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 611/1223] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:52:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:35.335806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84055f-53d2-4c06-bea3-683e9148eacf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:35.383498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:35.447221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84055f-53d2-4c06-bea3-683e9148eacf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:35.484850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba0df9-736f-4a7a-90e6-4bc777388086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:35.485635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba0df9-736f-4a7a-90e6-4bc777388086 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:35.485918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba0df9-736f-4a7a-90e6-4bc777388086 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:35.486194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba0df9-736f-4a7a-90e6-4bc777388086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:35.496932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba0df9-736f-4a7a-90e6-4bc777388086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:35.498554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba0df9-736f-4a7a-90e6-4bc777388086 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-faba0df9-736f-4a7a-90e6-4bc777388086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:35.516177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:35.517725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:35.519099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:35.519099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:35.519099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:35.519099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:35.519099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 613/1224] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:52:35.524764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1312f-5aed-4b31-a60b-3d537a5459e0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:35.633852 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1312f-5aed-4b31-a60b-3d537a5459e0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:35.669444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591424c-66c3-48bc-84fd-ec330c246cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:35.670140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591424c-66c3-48bc-84fd-ec330c246cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:35.670426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591424c-66c3-48bc-84fd-ec330c246cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:35.670705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591424c-66c3-48bc-84fd-ec330c246cc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:35.671425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591424c-66c3-48bc-84fd-ec330c246cc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:35.671764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591424c-66c3-48bc-84fd-ec330c246cc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:35.673134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591424c-66c3-48bc-84fd-ec330c246cc4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:35.685250 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 612/1225] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:35.691364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba0df9-736f-4a7a-90e6-4bc777388086 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:35.794691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:35.803304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba0df9-736f-4a7a-90e6-4bc777388086 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:35.843757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde7236-d996-40d1-be53-ef77be242f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:35.844603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde7236-d996-40d1-be53-ef77be242f92 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:35.844895 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde7236-d996-40d1-be53-ef77be242f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:35.845171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde7236-d996-40d1-be53-ef77be242f92 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:35.853092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde7236-d996-40d1-be53-ef77be242f92 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:35.854746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde7236-d996-40d1-be53-ef77be242f92 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-bbde7236-d996-40d1-be53-ef77be242f92 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:52:35.871719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:35.873433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:35.879209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:35.879209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:35.879209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:52:35.879209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:52:35.879209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:52:35.879209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:52:35.879209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:52:35.879209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:52:35.879209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:52:35.879209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:35.879209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 614/1226] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:35.879860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591424c-66c3-48bc-84fd-ec330c246cc4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:35.931782 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:35.994748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591424c-66c3-48bc-84fd-ec330c246cc4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:36.035671 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ed9c0-47ea-43e6-a056-d63a85d3fb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:36.036369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ed9c0-47ea-43e6-a056-d63a85d3fb3d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:36.036703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ed9c0-47ea-43e6-a056-d63a85d3fb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:36.036953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ed9c0-47ea-43e6-a056-d63a85d3fb3d None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:36.037952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ed9c0-47ea-43e6-a056-d63a85d3fb3d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:36.038274 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ed9c0-47ea-43e6-a056-d63a85d3fb3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:36.039996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ed9c0-47ea-43e6-a056-d63a85d3fb3d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:36.061902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 613/1227] 10.222.0.65 () {56 vars in 1143 bytes} [Thu Jun 23 10:52:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:36.067696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde7236-d996-40d1-be53-ef77be242f92 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:36.173801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde7236-d996-40d1-be53-ef77be242f92 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:36.208574 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b37dc8-d783-498f-a327-84d90a84d8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:36.209300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b37dc8-d783-498f-a327-84d90a84d8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:36.209611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b37dc8-d783-498f-a327-84d90a84d8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:36.209867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b37dc8-d783-498f-a327-84d90a84d8b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:36.220085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b37dc8-d783-498f-a327-84d90a84d8b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:36.221536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b37dc8-d783-498f-a327-84d90a84d8b2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-91b37dc8-d783-498f-a327-84d90a84d8b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:36.240065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:36.241190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:36.242230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:36.242230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:36.242230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:36.242230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:36.242230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 615/1228] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:36.247269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ed9c0-47ea-43e6-a056-d63a85d3fb3d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:36.352435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ed9c0-47ea-43e6-a056-d63a85d3fb3d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:36.354801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:36.388757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ca8a8-a60f-467a-b440-faa18001c350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:36.389423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ca8a8-a60f-467a-b440-faa18001c350 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:36.389702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ca8a8-a60f-467a-b440-faa18001c350 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:36.389970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ca8a8-a60f-467a-b440-faa18001c350 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:36.390684 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ca8a8-a60f-467a-b440-faa18001c350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:36.390990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ca8a8-a60f-467a-b440-faa18001c350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:36.392406 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ca8a8-a60f-467a-b440-faa18001c350 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:36.404468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 614/1229] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:36.410306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b37dc8-d783-498f-a327-84d90a84d8b2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:36.421062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:36.518186 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b37dc8-d783-498f-a327-84d90a84d8b2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:36.556415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f1b67-3bca-40b4-b9b1-d7661deca66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:36.557206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f1b67-3bca-40b4-b9b1-d7661deca66c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:36.557533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f1b67-3bca-40b4-b9b1-d7661deca66c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:36.557808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f1b67-3bca-40b4-b9b1-d7661deca66c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:36.565019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f1b67-3bca-40b4-b9b1-d7661deca66c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:36.566523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f1b67-3bca-40b4-b9b1-d7661deca66c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5a6f1b67-3bca-40b4-b9b1-d7661deca66c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:52:36.582171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:36.583280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:36.584442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:52:36.584442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:52:36.584442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:52:36.584442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:52:36.584442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:52:36.584442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:52:36.584442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:52:36.584442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:36.584442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 616/1230] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:52:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:52:36.591492 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ca8a8-a60f-467a-b440-faa18001c350 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:36.698664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ca8a8-a60f-467a-b440-faa18001c350 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:36.733657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d487a-ead0-43eb-ad6e-990df3a02377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:36.734371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d487a-ead0-43eb-ad6e-990df3a02377 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:36.734659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d487a-ead0-43eb-ad6e-990df3a02377 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:36.734925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d487a-ead0-43eb-ad6e-990df3a02377 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:36.735731 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d487a-ead0-43eb-ad6e-990df3a02377 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:36.736045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d487a-ead0-43eb-ad6e-990df3a02377 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:36.737556 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d487a-ead0-43eb-ad6e-990df3a02377 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:36.750554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 615/1231] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:52:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:52:36.756504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f1b67-3bca-40b4-b9b1-d7661deca66c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:36.825813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:36.861651 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f1b67-3bca-40b4-b9b1-d7661deca66c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:36.899718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37690c6c-96eb-4a9e-b9e7-a3eadc8791f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:36.900677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37690c6c-96eb-4a9e-b9e7-a3eadc8791f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:36.901004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37690c6c-96eb-4a9e-b9e7-a3eadc8791f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:36.901301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37690c6c-96eb-4a9e-b9e7-a3eadc8791f9 None admin] PATH_INFO: `/v3/projects/8458544c021049c587856343e8001429/users/18fc0e7956b14a49beab025922e8ff81/roles/b29c780ef8924837b89e1f06648a9563` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:36.917583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37690c6c-96eb-4a9e-b9e7-a3eadc8791f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8458544c021049c587856343e8001429, user_id=18fc0e7956b14a49beab025922e8ff81, role_id=b29c780ef8924837b89e1f06648a9563)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:36.919050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37690c6c-96eb-4a9e-b9e7-a3eadc8791f9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:36.932970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 617/1232] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:52:36 2022] PUT /identity/v3/projects/8458544c021049c587856343e8001429/users/18fc0e7956b14a49beab025922e8ff81/roles/b29c780ef8924837b89e1f06648a9563 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:52:39.053182 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e302b-a46d-493f-af2e-1214672e9326 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:39.053775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e302b-a46d-493f-af2e-1214672e9326 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:39.053775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e302b-a46d-493f-af2e-1214672e9326 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:39.055125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 616/1233] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:52:39.061008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ced95-34a5-492f-9097-66bc7764202a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:39.061423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ced95-34a5-492f-9097-66bc7764202a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:39.061567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ced95-34a5-492f-9097-66bc7764202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:39.079632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ee1ced95-34a5-492f-9097-66bc7764202a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:52:39.080885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ee1ced95-34a5-492f-9097-66bc7764202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:52:39.155245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 618/1234] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:52:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:52:39.179760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e302b-a46d-493f-af2e-1214672e9326 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:39.205737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:39.287788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e302b-a46d-493f-af2e-1214672e9326 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:39.326199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c0f87-fc69-42a2-bba5-27a82b008ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:39.327262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c0f87-fc69-42a2-bba5-27a82b008ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:39.327534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c0f87-fc69-42a2-bba5-27a82b008ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:39.328716 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c0f87-fc69-42a2-bba5-27a82b008ee2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:39.338913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c0f87-fc69-42a2-bba5-27a82b008ee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-986c0f87-fc69-42a2-bba5-27a82b008ee2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:52:39.346660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:39.347977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:39.349096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 617/1235] 10.222.0.65 () {56 vars in 1102 bytes} [Thu Jun 23 10:52:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:39.353801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ced95-34a5-492f-9097-66bc7764202a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:39.403281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:39.461860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ced95-34a5-492f-9097-66bc7764202a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:39.497680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8bdedb-66a3-4b12-af3b-fba4560660f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:39.498874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bdedb-66a3-4b12-af3b-fba4560660f7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:39.499162 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bdedb-66a3-4b12-af3b-fba4560660f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:39.499441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bdedb-66a3-4b12-af3b-fba4560660f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:39.500226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8bdedb-66a3-4b12-af3b-fba4560660f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:39.500538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8bdedb-66a3-4b12-af3b-fba4560660f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:39.501900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8bdedb-66a3-4b12-af3b-fba4560660f7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:39.514987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 619/1236] 10.222.0.65 () {56 vars in 1103 bytes} [Thu Jun 23 10:52:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:52:39.516324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c0f87-fc69-42a2-bba5-27a82b008ee2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:39.603014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:39.620284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c0f87-fc69-42a2-bba5-27a82b008ee2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:39.659300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e0839-afbc-4323-a838-462d0dc71022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:39.660189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e0839-afbc-4323-a838-462d0dc71022 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:39.660577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e0839-afbc-4323-a838-462d0dc71022 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:39.660963 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e0839-afbc-4323-a838-462d0dc71022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:39.674162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e0839-afbc-4323-a838-462d0dc71022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:39.676072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e0839-afbc-4323-a838-462d0dc71022 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-df1e0839-afbc-4323-a838-462d0dc71022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:39.694999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:39.696021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:39.696940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:39.696940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:39.696940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:39.696940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:39.696940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 618/1237] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:39.702691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8bdedb-66a3-4b12-af3b-fba4560660f7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:39.809304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8bdedb-66a3-4b12-af3b-fba4560660f7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:39.818884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:39.847447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009f077-58e3-4131-a80b-52a15d9dbb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:39.848172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009f077-58e3-4131-a80b-52a15d9dbb38 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:39.848492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009f077-58e3-4131-a80b-52a15d9dbb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:39.848810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009f077-58e3-4131-a80b-52a15d9dbb38 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:39.849650 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009f077-58e3-4131-a80b-52a15d9dbb38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:39.850101 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009f077-58e3-4131-a80b-52a15d9dbb38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:39.851625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009f077-58e3-4131-a80b-52a15d9dbb38 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:39.871065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 620/1238] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:52:39.873842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e0839-afbc-4323-a838-462d0dc71022 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:39.986023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e0839-afbc-4323-a838-462d0dc71022 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:40.022658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1c227-7dcc-49f6-94a5-4ea24f3ee45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:40.023462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1c227-7dcc-49f6-94a5-4ea24f3ee45f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:40.023771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1c227-7dcc-49f6-94a5-4ea24f3ee45f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:40.024047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1c227-7dcc-49f6-94a5-4ea24f3ee45f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:40.026824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:40.032636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1c227-7dcc-49f6-94a5-4ea24f3ee45f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:40.034369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1c227-7dcc-49f6-94a5-4ea24f3ee45f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-82c1c227-7dcc-49f6-94a5-4ea24f3ee45f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:52:40.050262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:40.051234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:40.056947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:40.056947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:40.056947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:52:40.056947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:52:40.056947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:52:40.056947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:52:40.056947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:52:40.056947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:52:40.056947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:52:40.056947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:40.056947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 619/1239] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:40.057560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009f077-58e3-4131-a80b-52a15d9dbb38 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:40.164391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009f077-58e3-4131-a80b-52a15d9dbb38 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:40.201329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2838ca-0f08-4a4b-8dab-a968d6b1ab11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:40.202060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2838ca-0f08-4a4b-8dab-a968d6b1ab11 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:40.202427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2838ca-0f08-4a4b-8dab-a968d6b1ab11 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:40.202625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2838ca-0f08-4a4b-8dab-a968d6b1ab11 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:40.203485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2838ca-0f08-4a4b-8dab-a968d6b1ab11 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:40.203825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2838ca-0f08-4a4b-8dab-a968d6b1ab11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:40.205300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2838ca-0f08-4a4b-8dab-a968d6b1ab11 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:40.228343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 621/1240] 10.222.0.65 () {56 vars in 1143 bytes} [Thu Jun 23 10:52:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:40.229643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1c227-7dcc-49f6-94a5-4ea24f3ee45f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:40.340388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1c227-7dcc-49f6-94a5-4ea24f3ee45f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:40.375878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b1fb8-6179-4ac6-b5a7-22e34aacbd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:40.376662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b1fb8-6179-4ac6-b5a7-22e34aacbd19 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:40.377014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b1fb8-6179-4ac6-b5a7-22e34aacbd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:40.377388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b1fb8-6179-4ac6-b5a7-22e34aacbd19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:40.448026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:40.456080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b1fb8-6179-4ac6-b5a7-22e34aacbd19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:40.457746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b1fb8-6179-4ac6-b5a7-22e34aacbd19 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:40.569876 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 620/1241] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:52:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:52:40.576415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2838ca-0f08-4a4b-8dab-a968d6b1ab11 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:40.608223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:40.691089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2838ca-0f08-4a4b-8dab-a968d6b1ab11 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:40.728129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9d605-3b6a-499f-b8ff-a6f2282d1739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:40.728861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9d605-3b6a-499f-b8ff-a6f2282d1739 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:40.729172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9d605-3b6a-499f-b8ff-a6f2282d1739 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:40.729428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9d605-3b6a-499f-b8ff-a6f2282d1739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:40.740499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9d605-3b6a-499f-b8ff-a6f2282d1739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:40.741945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9d605-3b6a-499f-b8ff-a6f2282d1739 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ecb9d605-3b6a-499f-b8ff-a6f2282d1739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:40.760011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:40.761226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:40.762552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:40.762552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:40.762552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:40.762552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:40.762552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 622/1242] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:40.767520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b1fb8-6179-4ac6-b5a7-22e34aacbd19 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:40.869700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:40.877257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b1fb8-6179-4ac6-b5a7-22e34aacbd19 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:40.916082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23714a61-14ae-4414-9079-53ce50fee31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:40.916861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23714a61-14ae-4414-9079-53ce50fee31f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:40.917153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23714a61-14ae-4414-9079-53ce50fee31f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:40.917523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23714a61-14ae-4414-9079-53ce50fee31f None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:40.918520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23714a61-14ae-4414-9079-53ce50fee31f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:40.918884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23714a61-14ae-4414-9079-53ce50fee31f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:40.921193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23714a61-14ae-4414-9079-53ce50fee31f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:40.936295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 621/1243] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:40.943190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9d605-3b6a-499f-b8ff-a6f2282d1739 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:41.029828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:41.053821 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9d605-3b6a-499f-b8ff-a6f2282d1739 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:41.091366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4821df5-79e8-46e1-aaac-d8ae3227bd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:41.092770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4821df5-79e8-46e1-aaac-d8ae3227bd85 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:41.093052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4821df5-79e8-46e1-aaac-d8ae3227bd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:41.093352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4821df5-79e8-46e1-aaac-d8ae3227bd85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:41.100675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4821df5-79e8-46e1-aaac-d8ae3227bd85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:41.102242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4821df5-79e8-46e1-aaac-d8ae3227bd85 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c4821df5-79e8-46e1-aaac-d8ae3227bd85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:52:41.118060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:41.119087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:41.120017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:52:41.120017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:52:41.120017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:52:41.120017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:52:41.120017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:52:41.120017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:52:41.120017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:52:41.120017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:41.120017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 623/1244] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:52:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:41.125557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23714a61-14ae-4414-9079-53ce50fee31f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:41.233944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23714a61-14ae-4414-9079-53ce50fee31f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:41.270568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeeb959-c64d-4aa8-a602-0b188893d32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:41.271331 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeb959-c64d-4aa8-a602-0b188893d32e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:41.271622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeb959-c64d-4aa8-a602-0b188893d32e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:41.271954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeb959-c64d-4aa8-a602-0b188893d32e None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:41.272846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeeb959-c64d-4aa8-a602-0b188893d32e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:41.273164 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeeb959-c64d-4aa8-a602-0b188893d32e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:41.274593 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeeb959-c64d-4aa8-a602-0b188893d32e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:41.288079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 622/1245] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:52:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:41.294269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4821df5-79e8-46e1-aaac-d8ae3227bd85 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:41.409627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4821df5-79e8-46e1-aaac-d8ae3227bd85 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:41.447740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa5a9e-abe5-4295-9339-bde402be26e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:41.448434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa5a9e-abe5-4295-9339-bde402be26e1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:41.448716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa5a9e-abe5-4295-9339-bde402be26e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:41.448991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa5a9e-abe5-4295-9339-bde402be26e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:41.449886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa5a9e-abe5-4295-9339-bde402be26e1 None admin] RBAC: Adding query filter params (role.id=b29c780ef8924837b89e1f06648a9563, scope.project.id=8458544c021049c587856343e8001429, user.id=18fc0e7956b14a49beab025922e8ff81) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:41.450202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa5a9e-abe5-4295-9339-bde402be26e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:41.452196 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa5a9e-abe5-4295-9339-bde402be26e1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:41.458747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 624/1246] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:52:41 2022] GET /identity/v3/role_assignments?user.id=18fc0e7956b14a49beab025922e8ff81&scope.project.id=8458544c021049c587856343e8001429&role.id=b29c780ef8924837b89e1f06648a9563 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:43.605181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8149bc-ad29-49ed-b5d6-57be32e33adc None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:43.605735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8149bc-ad29-49ed-b5d6-57be32e33adc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:43.605735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8149bc-ad29-49ed-b5d6-57be32e33adc None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:43.607252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 623/1247] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:52:43.613021 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cde1a-6223-47ec-b643-7ac417daedaa None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:43.613865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cde1a-6223-47ec-b643-7ac417daedaa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:43.613865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cde1a-6223-47ec-b643-7ac417daedaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:43.619031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:43.632323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a13cde1a-6223-47ec-b643-7ac417daedaa None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:52:43.633529 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a13cde1a-6223-47ec-b643-7ac417daedaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:52:43.705913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 625/1248] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:52:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:52:43.730728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8149bc-ad29-49ed-b5d6-57be32e33adc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:43.752595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:43.849796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8149bc-ad29-49ed-b5d6-57be32e33adc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:43.896378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b340f7-0228-4857-9cd3-765743476e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:43.897516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b340f7-0228-4857-9cd3-765743476e63 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:43.897972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b340f7-0228-4857-9cd3-765743476e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:43.898513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b340f7-0228-4857-9cd3-765743476e63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:43.913667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b340f7-0228-4857-9cd3-765743476e63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-23b340f7-0228-4857-9cd3-765743476e63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:52:43.923042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:43.924872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:43.926047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 624/1249] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:52:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:43.930686 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cde1a-6223-47ec-b643-7ac417daedaa None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:44.044277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cde1a-6223-47ec-b643-7ac417daedaa None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:44.076930 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:44.082782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d22c6-cd3d-4549-9494-5197c8a14446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:44.083464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d22c6-cd3d-4549-9494-5197c8a14446 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:44.083781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d22c6-cd3d-4549-9494-5197c8a14446 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:44.084056 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d22c6-cd3d-4549-9494-5197c8a14446 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:44.084803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d22c6-cd3d-4549-9494-5197c8a14446 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:44.085118 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d22c6-cd3d-4549-9494-5197c8a14446 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:44.086773 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d22c6-cd3d-4549-9494-5197c8a14446 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:44.097470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 626/1250] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:52:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:44.104455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b340f7-0228-4857-9cd3-765743476e63 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:44.193387 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:44.221310 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b340f7-0228-4857-9cd3-765743476e63 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:44.258919 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18ed39-799e-4e1f-ab94-43b6fe1af37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:44.259771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18ed39-799e-4e1f-ab94-43b6fe1af37e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:44.260069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18ed39-799e-4e1f-ab94-43b6fe1af37e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:44.260381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18ed39-799e-4e1f-ab94-43b6fe1af37e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:44.272425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18ed39-799e-4e1f-ab94-43b6fe1af37e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:44.273943 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18ed39-799e-4e1f-ab94-43b6fe1af37e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-aa18ed39-799e-4e1f-ab94-43b6fe1af37e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:44.291937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:44.293026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:44.298660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:44.298660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:44.298660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:44.298660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:44.298660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 625/1251] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:44.298968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d22c6-cd3d-4549-9494-5197c8a14446 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:44.405327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d22c6-cd3d-4549-9494-5197c8a14446 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:44.443465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cc1aa-ac2d-4d2c-a3aa-c67db5f92506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:44.444149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cc1aa-ac2d-4d2c-a3aa-c67db5f92506 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:44.444433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cc1aa-ac2d-4d2c-a3aa-c67db5f92506 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:44.444706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cc1aa-ac2d-4d2c-a3aa-c67db5f92506 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:44.445423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cc1aa-ac2d-4d2c-a3aa-c67db5f92506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:44.445741 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cc1aa-ac2d-4d2c-a3aa-c67db5f92506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:44.447096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cc1aa-ac2d-4d2c-a3aa-c67db5f92506 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:44.466160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 627/1252] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:44.467436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18ed39-799e-4e1f-ab94-43b6fe1af37e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:44.586910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18ed39-799e-4e1f-ab94-43b6fe1af37e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:44.613092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:44.621581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f389ece-437a-4e6b-b86d-36ffa47dccd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:44.622447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f389ece-437a-4e6b-b86d-36ffa47dccd0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:44.622694 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f389ece-437a-4e6b-b86d-36ffa47dccd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:44.622977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f389ece-437a-4e6b-b86d-36ffa47dccd0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:44.632388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f389ece-437a-4e6b-b86d-36ffa47dccd0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:44.633828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f389ece-437a-4e6b-b86d-36ffa47dccd0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4f389ece-437a-4e6b-b86d-36ffa47dccd0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:52:44.648815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:44.649838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:44.650877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:44.650877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:44.650877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:52:44.650877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:52:44.650877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:52:44.650877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:52:44.650877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:52:44.650877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:52:44.650877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:52:44.650877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:44.650877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 626/1253] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:44.656083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cc1aa-ac2d-4d2c-a3aa-c67db5f92506 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:44.717962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:44.774601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cc1aa-ac2d-4d2c-a3aa-c67db5f92506 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:44.811115 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d5913-770c-466e-82c9-1bd7f8da1bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:44.811919 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d5913-770c-466e-82c9-1bd7f8da1bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:44.812215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d5913-770c-466e-82c9-1bd7f8da1bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:44.812406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d5913-770c-466e-82c9-1bd7f8da1bb8 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:44.813267 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d5913-770c-466e-82c9-1bd7f8da1bb8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:44.813570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d5913-770c-466e-82c9-1bd7f8da1bb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:44.815037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d5913-770c-466e-82c9-1bd7f8da1bb8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:44.834033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 628/1254] 10.222.0.65 () {56 vars in 1143 bytes} [Thu Jun 23 10:52:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:52:44.840301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f389ece-437a-4e6b-b86d-36ffa47dccd0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:44.958517 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f389ece-437a-4e6b-b86d-36ffa47dccd0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:44.995283 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a77b7a-699a-42bd-a004-2c2ffecf161a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:44.995925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a77b7a-699a-42bd-a004-2c2ffecf161a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:44.996212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a77b7a-699a-42bd-a004-2c2ffecf161a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:44.996490 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a77b7a-699a-42bd-a004-2c2ffecf161a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:45.058360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:45.071433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a77b7a-699a-42bd-a004-2c2ffecf161a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:45.072984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a77b7a-699a-42bd-a004-2c2ffecf161a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:45.192920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 627/1255] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:52:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:52:45.198909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d5913-770c-466e-82c9-1bd7f8da1bb8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:45.305086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d5913-770c-466e-82c9-1bd7f8da1bb8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:45.318877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:45.341557 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be847fb0-875f-4892-bbe2-839a5f86d882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:45.342897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be847fb0-875f-4892-bbe2-839a5f86d882 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:45.343181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be847fb0-875f-4892-bbe2-839a5f86d882 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:45.343460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be847fb0-875f-4892-bbe2-839a5f86d882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:45.354090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be847fb0-875f-4892-bbe2-839a5f86d882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:45.355584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be847fb0-875f-4892-bbe2-839a5f86d882 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-be847fb0-875f-4892-bbe2-839a5f86d882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:45.374039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:45.375227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:45.376497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:45.376497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:45.376497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:45.376497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:45.376497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 629/1256] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:52:45.381153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a77b7a-699a-42bd-a004-2c2ffecf161a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:45.470630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:45.485468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a77b7a-699a-42bd-a004-2c2ffecf161a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:45.522770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d919a-9b44-4346-83c9-da0666869c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:45.523488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d919a-9b44-4346-83c9-da0666869c5f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:45.523821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d919a-9b44-4346-83c9-da0666869c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:45.524144 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d919a-9b44-4346-83c9-da0666869c5f None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:45.524954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d919a-9b44-4346-83c9-da0666869c5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:45.525263 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d919a-9b44-4346-83c9-da0666869c5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:45.526793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d919a-9b44-4346-83c9-da0666869c5f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:45.538660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 628/1257] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:45.544245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be847fb0-875f-4892-bbe2-839a5f86d882 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:45.661717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be847fb0-875f-4892-bbe2-839a5f86d882 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:45.697929 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1ca49-3d7c-4ecb-817f-d37d2b01b304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:45.698727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1ca49-3d7c-4ecb-817f-d37d2b01b304 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:45.699019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1ca49-3d7c-4ecb-817f-d37d2b01b304 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:45.699293 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1ca49-3d7c-4ecb-817f-d37d2b01b304 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:45.706606 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1ca49-3d7c-4ecb-817f-d37d2b01b304 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:45.708083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1ca49-3d7c-4ecb-817f-d37d2b01b304 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-12f1ca49-3d7c-4ecb-817f-d37d2b01b304 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:52:45.722401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:45.723524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:45.729208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:52:45.729208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:52:45.729208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:52:45.729208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:52:45.729208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:52:45.729208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:52:45.729208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:52:45.729208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:45.729208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 630/1258] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:52:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:45.729765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d919a-9b44-4346-83c9-da0666869c5f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:45.834468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d919a-9b44-4346-83c9-da0666869c5f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:45.875497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b91db7-06a5-45ed-b651-f6e6038a9a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:45.876231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b91db7-06a5-45ed-b651-f6e6038a9a86 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:45.876512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b91db7-06a5-45ed-b651-f6e6038a9a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:45.876814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b91db7-06a5-45ed-b651-f6e6038a9a86 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:45.877781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b91db7-06a5-45ed-b651-f6e6038a9a86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:45.878060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b91db7-06a5-45ed-b651-f6e6038a9a86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:45.879829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b91db7-06a5-45ed-b651-f6e6038a9a86 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:45.895898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 629/1259] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:52:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:45.902647 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1ca49-3d7c-4ecb-817f-d37d2b01b304 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:45.933909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:46.014982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1ca49-3d7c-4ecb-817f-d37d2b01b304 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:46.050945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8c75c-3b0a-4120-ae22-7304bfb0db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:46.051610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8c75c-3b0a-4120-ae22-7304bfb0db47 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:46.051923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8c75c-3b0a-4120-ae22-7304bfb0db47 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:46.052203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8c75c-3b0a-4120-ae22-7304bfb0db47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:46.053052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8c75c-3b0a-4120-ae22-7304bfb0db47 None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=8458544c021049c587856343e8001429, user.id=18fc0e7956b14a49beab025922e8ff81) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:46.053367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8c75c-3b0a-4120-ae22-7304bfb0db47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:46.054823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8c75c-3b0a-4120-ae22-7304bfb0db47 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:46.061037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 631/1260] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:52:45 2022] GET /identity/v3/role_assignments?user.id=18fc0e7956b14a49beab025922e8ff81&scope.project.id=8458544c021049c587856343e8001429&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:48.069639 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905522d-ba8c-4e79-8157-984d91bdb817 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:48.070147 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905522d-ba8c-4e79-8157-984d91bdb817 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:48.070265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905522d-ba8c-4e79-8157-984d91bdb817 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:48.071578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 630/1261] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:52:48.077363 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0dd14-e886-417e-bf8e-110477beb7fe None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:48.077752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0dd14-e886-417e-bf8e-110477beb7fe None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:48.077920 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0dd14-e886-417e-bf8e-110477beb7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:48.094833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8bc0dd14-e886-417e-bf8e-110477beb7fe None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:52:48.096589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8bc0dd14-e886-417e-bf8e-110477beb7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:52:48.160997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 632/1262] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:52:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:52:48.184200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905522d-ba8c-4e79-8157-984d91bdb817 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:48.195525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:48.285497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905522d-ba8c-4e79-8157-984d91bdb817 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:48.318881 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1839c1-bde7-4eea-a4d3-1d32d8376020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:48.319723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1839c1-bde7-4eea-a4d3-1d32d8376020 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:48.319958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1839c1-bde7-4eea-a4d3-1d32d8376020 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:48.320232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1839c1-bde7-4eea-a4d3-1d32d8376020 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:48.329933 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1839c1-bde7-4eea-a4d3-1d32d8376020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0a1839c1-bde7-4eea-a4d3-1d32d8376020 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:52:48.337378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:48.338479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:48.339544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 631/1263] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:52:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:48.344697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0dd14-e886-417e-bf8e-110477beb7fe None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:48.362793 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:48.440373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0dd14-e886-417e-bf8e-110477beb7fe None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:48.472536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55132fa9-5b29-45a4-b6d2-3da1951e72bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:48.473183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55132fa9-5b29-45a4-b6d2-3da1951e72bb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:48.473461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55132fa9-5b29-45a4-b6d2-3da1951e72bb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:48.473732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55132fa9-5b29-45a4-b6d2-3da1951e72bb None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:48.474462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55132fa9-5b29-45a4-b6d2-3da1951e72bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:48.474764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55132fa9-5b29-45a4-b6d2-3da1951e72bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:48.476141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55132fa9-5b29-45a4-b6d2-3da1951e72bb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:48.484441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 633/1264] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:52:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:52:48.490270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1839c1-bde7-4eea-a4d3-1d32d8376020 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:48.554510 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:48.587481 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1839c1-bde7-4eea-a4d3-1d32d8376020 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:48.620588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf43f6-86af-4a6a-9130-c5716299175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:48.621368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf43f6-86af-4a6a-9130-c5716299175e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:48.621651 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf43f6-86af-4a6a-9130-c5716299175e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:48.621928 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf43f6-86af-4a6a-9130-c5716299175e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:48.632119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf43f6-86af-4a6a-9130-c5716299175e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:48.633550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf43f6-86af-4a6a-9130-c5716299175e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6fcf43f6-86af-4a6a-9130-c5716299175e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:48.650011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:48.651085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:48.652258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:48.652258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:48.652258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:48.652258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:48.652258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 632/1265] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:48.657386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55132fa9-5b29-45a4-b6d2-3da1951e72bb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:48.743528 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:48.762339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55132fa9-5b29-45a4-b6d2-3da1951e72bb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:48.794145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4cd96-025b-48e2-b827-68fcff9a0674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:48.794790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4cd96-025b-48e2-b827-68fcff9a0674 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:48.795719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4cd96-025b-48e2-b827-68fcff9a0674 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:48.796001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4cd96-025b-48e2-b827-68fcff9a0674 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:48.796710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4cd96-025b-48e2-b827-68fcff9a0674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:48.797014 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4cd96-025b-48e2-b827-68fcff9a0674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:48.798358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4cd96-025b-48e2-b827-68fcff9a0674 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:48.809514 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 634/1266] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:48.815240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf43f6-86af-4a6a-9130-c5716299175e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:48.924270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf43f6-86af-4a6a-9130-c5716299175e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:48.943910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:48.959213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fae80-2470-4119-a374-af7e84144362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:48.960052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fae80-2470-4119-a374-af7e84144362 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:48.960341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fae80-2470-4119-a374-af7e84144362 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:48.960622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fae80-2470-4119-a374-af7e84144362 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:48.968669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fae80-2470-4119-a374-af7e84144362 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:48.970106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fae80-2470-4119-a374-af7e84144362 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0d5fae80-2470-4119-a374-af7e84144362 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:52:48.984134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:48.985285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:48.986640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:48.986640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:48.986640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:52:48.986640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:52:48.986640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:52:48.986640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:52:48.986640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:52:48.986640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:52:48.986640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:52:48.986640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:48.986640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 633/1267] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:48.991379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4cd96-025b-48e2-b827-68fcff9a0674 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:49.085818 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4cd96-025b-48e2-b827-68fcff9a0674 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:49.118807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1bd0f-b113-4986-9b46-139d6d9304a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:49.119461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1bd0f-b113-4986-9b46-139d6d9304a5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:49.119774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1bd0f-b113-4986-9b46-139d6d9304a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:49.120050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1bd0f-b113-4986-9b46-139d6d9304a5 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:49.120870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1bd0f-b113-4986-9b46-139d6d9304a5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:49.121178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1bd0f-b113-4986-9b46-139d6d9304a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:49.122582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1bd0f-b113-4986-9b46-139d6d9304a5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:49.130854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:49.140475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 635/1268] 10.222.0.65 () {56 vars in 1143 bytes} [Thu Jun 23 10:52:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:49.145980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fae80-2470-4119-a374-af7e84144362 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:49.241423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fae80-2470-4119-a374-af7e84144362 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:49.273832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1743dff-8d90-4802-8b75-f7d5c5b0e74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:49.274605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1743dff-8d90-4802-8b75-f7d5c5b0e74d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:49.274884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1743dff-8d90-4802-8b75-f7d5c5b0e74d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:49.275159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1743dff-8d90-4802-8b75-f7d5c5b0e74d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:49.285344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1743dff-8d90-4802-8b75-f7d5c5b0e74d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:49.286770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1743dff-8d90-4802-8b75-f7d5c5b0e74d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a1743dff-8d90-4802-8b75-f7d5c5b0e74d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:49.303110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:49.304218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:49.305208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:49.305208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:49.305208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:49.305208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:49.305208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 634/1269] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:49.309858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1bd0f-b113-4986-9b46-139d6d9304a5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:49.407183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1bd0f-b113-4986-9b46-139d6d9304a5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:49.443123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36894369-ef43-4af8-9d19-af6562d11df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:49.443803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36894369-ef43-4af8-9d19-af6562d11df9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:49.444090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36894369-ef43-4af8-9d19-af6562d11df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:49.444369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36894369-ef43-4af8-9d19-af6562d11df9 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:49.445089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36894369-ef43-4af8-9d19-af6562d11df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:49.445400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36894369-ef43-4af8-9d19-af6562d11df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:49.446754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36894369-ef43-4af8-9d19-af6562d11df9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:49.458284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 636/1270] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:49.463722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1743dff-8d90-4802-8b75-f7d5c5b0e74d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:49.494091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:49.561384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1743dff-8d90-4802-8b75-f7d5c5b0e74d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:49.593254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836742e-eaf7-4458-8aee-229880001d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:49.594027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836742e-eaf7-4458-8aee-229880001d13 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:49.594307 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836742e-eaf7-4458-8aee-229880001d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:49.594578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836742e-eaf7-4458-8aee-229880001d13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:49.601209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836742e-eaf7-4458-8aee-229880001d13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:49.602664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836742e-eaf7-4458-8aee-229880001d13 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d836742e-eaf7-4458-8aee-229880001d13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:52:49.618478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:49.619354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:49.620411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:52:49.620411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:52:49.620411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:52:49.620411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:52:49.620411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:52:49.620411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:52:49.620411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:52:49.620411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:49.620411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 635/1271] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:52:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:49.625483 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36894369-ef43-4af8-9d19-af6562d11df9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:49.681840 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:49.722787 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36894369-ef43-4af8-9d19-af6562d11df9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:49.754590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d970c-c3b0-4b37-91bf-da6b148631dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:49.755178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d970c-c3b0-4b37-91bf-da6b148631dc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:49.755462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d970c-c3b0-4b37-91bf-da6b148631dc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:49.755834 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d970c-c3b0-4b37-91bf-da6b148631dc None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:49.756616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d970c-c3b0-4b37-91bf-da6b148631dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:49.756927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d970c-c3b0-4b37-91bf-da6b148631dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:49.758340 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d970c-c3b0-4b37-91bf-da6b148631dc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:49.770355 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 637/1272] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:52:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:49.776090 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836742e-eaf7-4458-8aee-229880001d13 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:49.863636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:49.872686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836742e-eaf7-4458-8aee-229880001d13 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:49.915871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b6542-8e78-40f0-a6ce-d5d4403e5849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:49.916750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b6542-8e78-40f0-a6ce-d5d4403e5849 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:49.917031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b6542-8e78-40f0-a6ce-d5d4403e5849 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:49.917305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b6542-8e78-40f0-a6ce-d5d4403e5849 None admin] PATH_INFO: `/v3/projects/8458544c021049c587856343e8001429/users/18fc0e7956b14a49beab025922e8ff81/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:49.919229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b6542-8e78-40f0-a6ce-d5d4403e5849 None admin] RBAC: Authorizing `identity:create_grant(project_id=8458544c021049c587856343e8001429, user_id=18fc0e7956b14a49beab025922e8ff81, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:49.920681 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b6542-8e78-40f0-a6ce-d5d4403e5849 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:49.932729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 636/1273] 10.222.0.65 () {58 vars in 1439 bytes} [Thu Jun 23 10:52:49 2022] PUT /identity/v3/projects/8458544c021049c587856343e8001429/users/18fc0e7956b14a49beab025922e8ff81/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:52:51.924156 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45cc83-8907-4617-b6a9-0d3d2c7362ec None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:51.924687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45cc83-8907-4617-b6a9-0d3d2c7362ec None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:51.924836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45cc83-8907-4617-b6a9-0d3d2c7362ec None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:51.926770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 638/1274] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:52:51.932270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e97f7-4178-47a5-ac3b-3a930e1e99b0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:51.932683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e97f7-4178-47a5-ac3b-3a930e1e99b0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:51.932833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e97f7-4178-47a5-ac3b-3a930e1e99b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:51.949088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c16e97f7-4178-47a5-ac3b-3a930e1e99b0 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:52:51.950258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c16e97f7-4178-47a5-ac3b-3a930e1e99b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:52:52.014398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 637/1275] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:52:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:52:52.038298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45cc83-8907-4617-b6a9-0d3d2c7362ec None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:52.140677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45cc83-8907-4617-b6a9-0d3d2c7362ec None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:52.158556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:52.174892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1404b-e87d-44f9-8644-e6555d5e0e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:52.175626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1404b-e87d-44f9-8644-e6555d5e0e52 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:52.175945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1404b-e87d-44f9-8644-e6555d5e0e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:52.176225 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1404b-e87d-44f9-8644-e6555d5e0e52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:52.186061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1404b-e87d-44f9-8644-e6555d5e0e52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-38d1404b-e87d-44f9-8644-e6555d5e0e52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:52:52.193692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:52.194811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:52.195901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 639/1276] 10.222.0.65 () {56 vars in 1096 bytes} [Thu Jun 23 10:52:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:52.200957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e97f7-4178-47a5-ac3b-3a930e1e99b0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:52.280094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:52.298842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e97f7-4178-47a5-ac3b-3a930e1e99b0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:52.332002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7466e7-9165-47f8-b99b-44da79d6c1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:52.332658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7466e7-9165-47f8-b99b-44da79d6c1da None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:52.332942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7466e7-9165-47f8-b99b-44da79d6c1da None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:52.333218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7466e7-9165-47f8-b99b-44da79d6c1da None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:52.333959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7466e7-9165-47f8-b99b-44da79d6c1da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:52.334267 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7466e7-9165-47f8-b99b-44da79d6c1da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:52.335639 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7466e7-9165-47f8-b99b-44da79d6c1da None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:52.343437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 638/1277] 10.222.0.65 () {56 vars in 1099 bytes} [Thu Jun 23 10:52:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:52.348927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1404b-e87d-44f9-8644-e6555d5e0e52 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:52.445418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1404b-e87d-44f9-8644-e6555d5e0e52 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:52.476994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668e0cd-3a11-44ed-bd42-5dc312fb8760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:52.477761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668e0cd-3a11-44ed-bd42-5dc312fb8760 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:52.478038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668e0cd-3a11-44ed-bd42-5dc312fb8760 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:52.478308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668e0cd-3a11-44ed-bd42-5dc312fb8760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:52.488410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668e0cd-3a11-44ed-bd42-5dc312fb8760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:52.490081 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668e0cd-3a11-44ed-bd42-5dc312fb8760 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7668e0cd-3a11-44ed-bd42-5dc312fb8760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:52.507000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:52.508080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:52.509198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:52.509198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:52.509198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:52.509198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:52.509198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 640/1278] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:52.513719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7466e7-9165-47f8-b99b-44da79d6c1da None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:52.611175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7466e7-9165-47f8-b99b-44da79d6c1da None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:52.643527 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088efd6d-b493-4bd7-be1e-43d72857de7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:52.644198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088efd6d-b493-4bd7-be1e-43d72857de7d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:52.644482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088efd6d-b493-4bd7-be1e-43d72857de7d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:52.644760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088efd6d-b493-4bd7-be1e-43d72857de7d None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:52.645471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088efd6d-b493-4bd7-be1e-43d72857de7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:52.645781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088efd6d-b493-4bd7-be1e-43d72857de7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:52.647136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088efd6d-b493-4bd7-be1e-43d72857de7d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:52.658420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 639/1279] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:52.664055 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668e0cd-3a11-44ed-bd42-5dc312fb8760 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:52.694733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:52.762203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668e0cd-3a11-44ed-bd42-5dc312fb8760 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:52.793532 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405aac9-7ab0-4e76-a58a-1a76277854b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:52.794313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405aac9-7ab0-4e76-a58a-1a76277854b8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:52.794596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405aac9-7ab0-4e76-a58a-1a76277854b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:52.794868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405aac9-7ab0-4e76-a58a-1a76277854b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:52.802536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405aac9-7ab0-4e76-a58a-1a76277854b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:52.803974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405aac9-7ab0-4e76-a58a-1a76277854b8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1405aac9-7ab0-4e76-a58a-1a76277854b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:52:52.818236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:52.819375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:52.820608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:52.820608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:52.820608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:52:52.820608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:52:52.820608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:52:52.820608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:52:52.820608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:52:52.820608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:52:52.820608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:52:52.820608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:52.820608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 641/1280] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:52.825229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088efd6d-b493-4bd7-be1e-43d72857de7d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:52.828953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:52.925477 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088efd6d-b493-4bd7-be1e-43d72857de7d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:52.957831 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff79462-78d7-4d8d-8448-9ff8a9d03c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:52.958491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff79462-78d7-4d8d-8448-9ff8a9d03c46 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:52.958773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff79462-78d7-4d8d-8448-9ff8a9d03c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:52.959048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff79462-78d7-4d8d-8448-9ff8a9d03c46 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:52.959900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff79462-78d7-4d8d-8448-9ff8a9d03c46 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:52.960207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff79462-78d7-4d8d-8448-9ff8a9d03c46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:52.961623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff79462-78d7-4d8d-8448-9ff8a9d03c46 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:52.979247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 640/1281] 10.222.0.65 () {56 vars in 1143 bytes} [Thu Jun 23 10:52:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:52.985645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405aac9-7ab0-4e76-a58a-1a76277854b8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:53.071497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:53.081166 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405aac9-7ab0-4e76-a58a-1a76277854b8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:53.114724 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203263b-0e71-4276-8357-56c0bf4cc2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:53.115378 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203263b-0e71-4276-8357-56c0bf4cc2ce None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:53.115683 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203263b-0e71-4276-8357-56c0bf4cc2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:53.115995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203263b-0e71-4276-8357-56c0bf4cc2ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:53.180347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203263b-0e71-4276-8357-56c0bf4cc2ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:53.181794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203263b-0e71-4276-8357-56c0bf4cc2ce None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:53.266136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:53.276314 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 642/1282] 10.222.0.65 () {58 vars in 1282 bytes} [Thu Jun 23 10:52:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:52:53.282644 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff79462-78d7-4d8d-8448-9ff8a9d03c46 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:53.340255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:53.377990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff79462-78d7-4d8d-8448-9ff8a9d03c46 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:53.410927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f07cd-f2da-438d-9b05-9d7a10b69a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:53.411740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f07cd-f2da-438d-9b05-9d7a10b69a84 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:53.412027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f07cd-f2da-438d-9b05-9d7a10b69a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:53.412303 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f07cd-f2da-438d-9b05-9d7a10b69a84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:53.421931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f07cd-f2da-438d-9b05-9d7a10b69a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:53.423364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f07cd-f2da-438d-9b05-9d7a10b69a84 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d32f07cd-f2da-438d-9b05-9d7a10b69a84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:52:53.440172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:53.441031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:53.441961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:52:53.441961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:52:53.441961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:52:53.441961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:53.441961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 641/1283] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:53.446878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203263b-0e71-4276-8357-56c0bf4cc2ce None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:53.543288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203263b-0e71-4276-8357-56c0bf4cc2ce None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:53.575097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc731d-2fe9-40b0-bd79-957bfe84d509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:53.575768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc731d-2fe9-40b0-bd79-957bfe84d509 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:53.576053 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc731d-2fe9-40b0-bd79-957bfe84d509 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:53.576839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc731d-2fe9-40b0-bd79-957bfe84d509 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:53.577553 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc731d-2fe9-40b0-bd79-957bfe84d509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:53.577859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc731d-2fe9-40b0-bd79-957bfe84d509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:53.579216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc731d-2fe9-40b0-bd79-957bfe84d509 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:53.589977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 643/1284] 10.222.0.65 () {56 vars in 1109 bytes} [Thu Jun 23 10:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:53.595989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f07cd-f2da-438d-9b05-9d7a10b69a84 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:53.690655 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f07cd-f2da-438d-9b05-9d7a10b69a84 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:53.704727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:53.724752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331c038-ff17-477b-88ff-96cf7421c380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:53.725528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331c038-ff17-477b-88ff-96cf7421c380 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:53.725808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331c038-ff17-477b-88ff-96cf7421c380 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:53.726082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331c038-ff17-477b-88ff-96cf7421c380 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:53.732487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331c038-ff17-477b-88ff-96cf7421c380 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:53.733910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331c038-ff17-477b-88ff-96cf7421c380 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e331c038-ff17-477b-88ff-96cf7421c380 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:52:53.747363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:53.748973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:53.754601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:52:53.754601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:52:53.754601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:52:53.754601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:52:53.754601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:52:53.754601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:52:53.754601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:52:53.754601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 23 10:52:53.754601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 642/1285] 10.222.0.65 () {56 vars in 1111 bytes} [Thu Jun 23 10:52:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:53.755230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc731d-2fe9-40b0-bd79-957bfe84d509 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:53.802710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:53.851070 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc731d-2fe9-40b0-bd79-957bfe84d509 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:53.885329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a845b-f733-4291-a26d-488982d85323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:53.885948 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a845b-f733-4291-a26d-488982d85323 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:53.886228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a845b-f733-4291-a26d-488982d85323 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:53.886526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a845b-f733-4291-a26d-488982d85323 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:53.887282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a845b-f733-4291-a26d-488982d85323 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:53.887590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a845b-f733-4291-a26d-488982d85323 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:53.889646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a845b-f733-4291-a26d-488982d85323 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:53.904077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 644/1286] 10.222.0.65 () {56 vars in 1148 bytes} [Thu Jun 23 10:52:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:53.910117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331c038-ff17-477b-88ff-96cf7421c380 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:54.008463 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331c038-ff17-477b-88ff-96cf7421c380 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:54.041688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d10101-b476-4082-819f-c53f0c3b9667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:54.042554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d10101-b476-4082-819f-c53f0c3b9667 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:54.042951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d10101-b476-4082-819f-c53f0c3b9667 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:54.043355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d10101-b476-4082-819f-c53f0c3b9667 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:54.044577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d10101-b476-4082-819f-c53f0c3b9667 None admin] RBAC: Adding query filter params (role.id=266f01c0faa946f0acfca0aac74f2461, scope.project.id=8458544c021049c587856343e8001429, user.id=18fc0e7956b14a49beab025922e8ff81) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:54.044900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d10101-b476-4082-819f-c53f0c3b9667 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:54.047007 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d10101-b476-4082-819f-c53f0c3b9667 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:54.053032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 643/1287] 10.222.0.65 () {56 vars in 1374 bytes} [Thu Jun 23 10:52:53 2022] GET /identity/v3/role_assignments?user.id=18fc0e7956b14a49beab025922e8ff81&scope.project.id=8458544c021049c587856343e8001429&role.id=266f01c0faa946f0acfca0aac74f2461 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:56.071552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca58b4-4674-411b-8660-a46866f5e248 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:56.072121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca58b4-4674-411b-8660-a46866f5e248 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:56.072121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca58b4-4674-411b-8660-a46866f5e248 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:56.073490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 645/1288] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:52:56.078545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc544f6a-1b3d-4f6d-aace-8193e02fa0ea None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:56.078938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc544f6a-1b3d-4f6d-aace-8193e02fa0ea None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:56.079093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc544f6a-1b3d-4f6d-aace-8193e02fa0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:56.096543 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dc544f6a-1b3d-4f6d-aace-8193e02fa0ea None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:52:56.097497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dc544f6a-1b3d-4f6d-aace-8193e02fa0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:52:56.126957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:56.163377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 644/1289] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:52:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:52:56.185979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca58b4-4674-411b-8660-a46866f5e248 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:56.283878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca58b4-4674-411b-8660-a46866f5e248 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:56.294085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:56.317646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3fb43-3f7c-4d49-abf5-230163f4e19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:56.318427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3fb43-3f7c-4d49-abf5-230163f4e19b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:56.318706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3fb43-3f7c-4d49-abf5-230163f4e19b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:56.318982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3fb43-3f7c-4d49-abf5-230163f4e19b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:56.319629 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3fb43-3f7c-4d49-abf5-230163f4e19b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:56.321008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3fb43-3f7c-4d49-abf5-230163f4e19b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-61a3fb43-3f7c-4d49-abf5-230163f4e19b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:52:56.332603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:52:56.333826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:52:56.334924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 646/1290] 10.222.0.65 () {56 vars in 1117 bytes} [Thu Jun 23 10:52:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:52:56.339593 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc544f6a-1b3d-4f6d-aace-8193e02fa0ea None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:56.436355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc544f6a-1b3d-4f6d-aace-8193e02fa0ea None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:56.469897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04597dad-1605-4b09-af64-1b7db37c79cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:56.470548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04597dad-1605-4b09-af64-1b7db37c79cb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:56.470835 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04597dad-1605-4b09-af64-1b7db37c79cb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:56.471119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04597dad-1605-4b09-af64-1b7db37c79cb None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:56.471847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04597dad-1605-4b09-af64-1b7db37c79cb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:56.472161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04597dad-1605-4b09-af64-1b7db37c79cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:56.473523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04597dad-1605-4b09-af64-1b7db37c79cb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:56.478339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 645/1291] 10.222.0.65 () {56 vars in 1116 bytes} [Thu Jun 23 10:52:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:56.483872 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3fb43-3f7c-4d49-abf5-230163f4e19b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:56.580188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3fb43-3f7c-4d49-abf5-230163f4e19b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:56.612782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee944e-bc13-4d50-8614-8fffe75c0081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:56.613368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee944e-bc13-4d50-8614-8fffe75c0081 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:56.613653 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee944e-bc13-4d50-8614-8fffe75c0081 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:56.613931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee944e-bc13-4d50-8614-8fffe75c0081 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:56.614903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee944e-bc13-4d50-8614-8fffe75c0081 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:52:56.615213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee944e-bc13-4d50-8614-8fffe75c0081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:56.616603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee944e-bc13-4d50-8614-8fffe75c0081 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:56.620934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 647/1292] 10.222.0.65 () {56 vars in 1116 bytes} [Thu Jun 23 10:52:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:58.619173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab117fe-f1c4-45c0-84d5-c59828bad44d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:58.619723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab117fe-f1c4-45c0-84d5-c59828bad44d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:58.619817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab117fe-f1c4-45c0-84d5-c59828bad44d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:58.621141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 646/1293] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:52:58.626365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8195f-8c5c-4145-a58c-0ed539e2d810 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:58.626780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8195f-8c5c-4145-a58c-0ed539e2d810 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:58.626914 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8195f-8c5c-4145-a58c-0ed539e2d810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:58.643578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-66c8195f-8c5c-4145-a58c-0ed539e2d810 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:52:58.644763 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-66c8195f-8c5c-4145-a58c-0ed539e2d810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:52:58.674253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:58.715494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 648/1294] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:52:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:52:58.738746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab117fe-f1c4-45c0-84d5-c59828bad44d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:52:58.769214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:58.837299 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab117fe-f1c4-45c0-84d5-c59828bad44d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:52:58.870068 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27d24f-2cd1-4088-b52d-e3179297908b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:52:58.870667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27d24f-2cd1-4088-b52d-e3179297908b None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:52:58.870973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27d24f-2cd1-4088-b52d-e3179297908b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:52:58.871247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27d24f-2cd1-4088-b52d-e3179297908b None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:52:58.872006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27d24f-2cd1-4088-b52d-e3179297908b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:52:58.873378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27d24f-2cd1-4088-b52d-e3179297908b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:52:58.895306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 647/1295] 10.222.0.65 () {60 vars in 1141 bytes} [Thu Jun 23 10:52:58 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:53:00.891016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439028b-fc4f-41dd-98ef-e0cdcc308fb3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:00.891700 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439028b-fc4f-41dd-98ef-e0cdcc308fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:00.891700 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439028b-fc4f-41dd-98ef-e0cdcc308fb3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:00.892951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 649/1296] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:53:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:00.898229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0a0a9-10a1-4ee1-94fe-9ce0c07f1119 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:00.898673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0a0a9-10a1-4ee1-94fe-9ce0c07f1119 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:00.898775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0a0a9-10a1-4ee1-94fe-9ce0c07f1119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:00.915808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-99e0a0a9-10a1-4ee1-94fe-9ce0c07f1119 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:00.916987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-99e0a0a9-10a1-4ee1-94fe-9ce0c07f1119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:00.987209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 648/1297] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:53:01.009765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439028b-fc4f-41dd-98ef-e0cdcc308fb3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:01.109138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439028b-fc4f-41dd-98ef-e0cdcc308fb3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:01.142876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f0a3e-c150-407c-950e-e4dc7fc3888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:01.143603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f0a3e-c150-407c-950e-e4dc7fc3888e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:01.143911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f0a3e-c150-407c-950e-e4dc7fc3888e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:01.144190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f0a3e-c150-407c-950e-e4dc7fc3888e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:01.144841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f0a3e-c150-407c-950e-e4dc7fc3888e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:01.146205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f0a3e-c150-407c-950e-e4dc7fc3888e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a13f0a3e-c150-407c-950e-e4dc7fc3888e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:01.158390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:01.159710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:53:01.160956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 650/1298] 10.222.0.65 () {56 vars in 1117 bytes} [Thu Jun 23 10:53:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:01.165627 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0a0a9-10a1-4ee1-94fe-9ce0c07f1119 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:01.183500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:01.262603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0a0a9-10a1-4ee1-94fe-9ce0c07f1119 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:01.295306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e604e-7a3c-42bd-a469-b64bb1ed93b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:01.295986 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e604e-7a3c-42bd-a469-b64bb1ed93b6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:01.296271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e604e-7a3c-42bd-a469-b64bb1ed93b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:01.296548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e604e-7a3c-42bd-a469-b64bb1ed93b6 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:01.297244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e604e-7a3c-42bd-a469-b64bb1ed93b6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:01.297551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e604e-7a3c-42bd-a469-b64bb1ed93b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:01.298909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e604e-7a3c-42bd-a469-b64bb1ed93b6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:01.303949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 649/1299] 10.222.0.65 () {56 vars in 1116 bytes} [Thu Jun 23 10:53:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:01.309716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f0a3e-c150-407c-950e-e4dc7fc3888e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:01.341013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:01.407378 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f0a3e-c150-407c-950e-e4dc7fc3888e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:01.439659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8233dfc-a9c6-4ca8-8115-0d2aabb56d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:01.440317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8233dfc-a9c6-4ca8-8115-0d2aabb56d74 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:01.440601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8233dfc-a9c6-4ca8-8115-0d2aabb56d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:01.440883 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8233dfc-a9c6-4ca8-8115-0d2aabb56d74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:01.441662 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8233dfc-a9c6-4ca8-8115-0d2aabb56d74 None admin] RBAC: Adding query filter params (interface=public, service_id=b2289e7732014777a8aca8405f5c91a1, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:01.441975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8233dfc-a9c6-4ca8-8115-0d2aabb56d74 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:01.443320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8233dfc-a9c6-4ca8-8115-0d2aabb56d74 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:01.448212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 651/1300] 10.222.0.65 () {56 vars in 1251 bytes} [Thu Jun 23 10:53:01 2022] GET /identity/v3/endpoints?service_id=b2289e7732014777a8aca8405f5c91a1&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:01.454164 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e604e-7a3c-42bd-a469-b64bb1ed93b6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:01.532478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:01.549958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e604e-7a3c-42bd-a469-b64bb1ed93b6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:01.582948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d839e4-f7a1-4bbf-904e-b7197d48958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:01.583595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d839e4-f7a1-4bbf-904e-b7197d48958d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:01.583905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d839e4-f7a1-4bbf-904e-b7197d48958d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:01.584184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d839e4-f7a1-4bbf-904e-b7197d48958d None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:01.584854 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d839e4-f7a1-4bbf-904e-b7197d48958d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:01.586204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d839e4-f7a1-4bbf-904e-b7197d48958d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:01.592876 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 650/1301] 10.222.0.65 () {56 vars in 1087 bytes} [Thu Jun 23 10:53:01 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 10:53:03.574300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d65ba-3fa8-43df-9aeb-28f9149fd3fe None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:03.574845 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d65ba-3fa8-43df-9aeb-28f9149fd3fe None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:03.574845 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d65ba-3fa8-43df-9aeb-28f9149fd3fe None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:03.576268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 652/1302] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:03.581378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2f9e0-e184-4a0f-921e-27dda1a1567d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:03.581788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2f9e0-e184-4a0f-921e-27dda1a1567d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:03.581940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2f9e0-e184-4a0f-921e-27dda1a1567d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:03.598850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-23d2f9e0-e184-4a0f-921e-27dda1a1567d None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:03.600073 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-23d2f9e0-e184-4a0f-921e-27dda1a1567d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:03.664875 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 651/1303] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:53:03.688193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d65ba-3fa8-43df-9aeb-28f9149fd3fe None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:03.781855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:03.788794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d65ba-3fa8-43df-9aeb-28f9149fd3fe None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:03.821342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2e6c8-aea5-4531-ac41-2e174cf49127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:03.822124 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2e6c8-aea5-4531-ac41-2e174cf49127 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:03.822408 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2e6c8-aea5-4531-ac41-2e174cf49127 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:03.822688 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2e6c8-aea5-4531-ac41-2e174cf49127 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:03.823335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2e6c8-aea5-4531-ac41-2e174cf49127 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:03.824763 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2e6c8-aea5-4531-ac41-2e174cf49127 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3ad2e6c8-aea5-4531-ac41-2e174cf49127 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:03.836455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:03.837632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 23 10:53:03.838781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 653/1304] 10.222.0.65 () {56 vars in 1117 bytes} [Thu Jun 23 10:53:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:03.844055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2f9e0-e184-4a0f-921e-27dda1a1567d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:03.918500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:03.941186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2f9e0-e184-4a0f-921e-27dda1a1567d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:03.974487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06faf5e-cd31-4d8d-a748-4bca4532c75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:03.975142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06faf5e-cd31-4d8d-a748-4bca4532c75a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:03.975430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06faf5e-cd31-4d8d-a748-4bca4532c75a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:03.975740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06faf5e-cd31-4d8d-a748-4bca4532c75a None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:03.976443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06faf5e-cd31-4d8d-a748-4bca4532c75a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:03.976754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06faf5e-cd31-4d8d-a748-4bca4532c75a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:03.978119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06faf5e-cd31-4d8d-a748-4bca4532c75a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:03.983163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 652/1305] 10.222.0.65 () {56 vars in 1116 bytes} [Thu Jun 23 10:53:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:03.988827 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2e6c8-aea5-4531-ac41-2e174cf49127 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:04.084667 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2e6c8-aea5-4531-ac41-2e174cf49127 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:04.117606 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8113282-fb47-4df6-9193-0c1676e68662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:04.118253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8113282-fb47-4df6-9193-0c1676e68662 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:04.118541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8113282-fb47-4df6-9193-0c1676e68662 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:04.118822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8113282-fb47-4df6-9193-0c1676e68662 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:04.119556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8113282-fb47-4df6-9193-0c1676e68662 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:04.120949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8113282-fb47-4df6-9193-0c1676e68662 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:04.138903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 654/1306] 10.222.0.65 () {60 vars in 1144 bytes} [Thu Jun 23 10:53:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:53:13.980578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=30 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:14.134657 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:06.390805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce35b58-1f2b-4388-9462-ad31ac4c421e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:06.391260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce35b58-1f2b-4388-9462-ad31ac4c421e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:06.391486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce35b58-1f2b-4388-9462-ad31ac4c421e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:06.393017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 653/1307] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:06.399098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c1bd1-eead-4e68-b368-955d31f9f29a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:06.399485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c1bd1-eead-4e68-b368-955d31f9f29a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:06.399746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c1bd1-eead-4e68-b368-955d31f9f29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:06.521219 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-353c1bd1-eead-4e68-b368-955d31f9f29a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:06.522433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-353c1bd1-eead-4e68-b368-955d31f9f29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:06.651686 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 655/1308] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:06.693884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a08392-707e-4e41-9061-97ec1da65039 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:06.694208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a08392-707e-4e41-9061-97ec1da65039 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:06.694805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a08392-707e-4e41-9061-97ec1da65039 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:06.696252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 654/1309] 10.222.0.65 () {54 vars in 964 bytes} [Thu Jun 23 10:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:06.702351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39be0e-b989-4e98-a2b7-5f960afccd53 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:06.702687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39be0e-b989-4e98-a2b7-5f960afccd53 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:06.702971 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39be0e-b989-4e98-a2b7-5f960afccd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:06.780786 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-df39be0e-b989-4e98-a2b7-5f960afccd53 None None] MFA Rules not processed for user `e96b136657864104a51900debd1fbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:06.782490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-df39be0e-b989-4e98-a2b7-5f960afccd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:06.803818 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:06.862503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 656/1310] 10.222.0.65 () {58 vars in 1061 bytes} [Thu Jun 23 10:54:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 10:54:06.871576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a08392-707e-4e41-9061-97ec1da65039 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:06.873337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-54a08392-707e-4e41-9061-97ec1da65039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:06.986170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a08392-707e-4e41-9061-97ec1da65039 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:07.061868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2958d-4ad1-4b4f-8e64-cffec88e52be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:07.062553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2958d-4ad1-4b4f-8e64-cffec88e52be service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:07.062837 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2958d-4ad1-4b4f-8e64-cffec88e52be service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:07.063113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2958d-4ad1-4b4f-8e64-cffec88e52be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:07.064784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c6d2958d-4ad1-4b4f-8e64-cffec88e52be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:07.125614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:07.183351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2958d-4ad1-4b4f-8e64-cffec88e52be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:07.184893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2958d-4ad1-4b4f-8e64-cffec88e52be service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:07.205580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 655/1311] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:54:09.613903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807a40f-131b-4133-a54e-ded0fc34c745 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:09.614427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807a40f-131b-4133-a54e-ded0fc34c745 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:09.614521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807a40f-131b-4133-a54e-ded0fc34c745 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:09.616017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 657/1312] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:09.621181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480531f6-0aa9-4b03-b157-e9abd3ea1784 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:09.621588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480531f6-0aa9-4b03-b157-e9abd3ea1784 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:09.621738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480531f6-0aa9-4b03-b157-e9abd3ea1784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:09.698606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-480531f6-0aa9-4b03-b157-e9abd3ea1784 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:09.699858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-480531f6-0aa9-4b03-b157-e9abd3ea1784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:09.800578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 656/1313] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:09.863068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb83ec4-7305-4aa6-b628-872cbbe8ece8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:09.863457 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb83ec4-7305-4aa6-b628-872cbbe8ece8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:09.863842 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb83ec4-7305-4aa6-b628-872cbbe8ece8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:09.865865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 658/1314] 10.222.0.65 () {54 vars in 964 bytes} [Thu Jun 23 10:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:09.872744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208de371-afe4-4b06-aad1-f14f231c2563 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:09.873035 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208de371-afe4-4b06-aad1-f14f231c2563 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:09.873336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208de371-afe4-4b06-aad1-f14f231c2563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:09.884662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:09.962939 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-208de371-afe4-4b06-aad1-f14f231c2563 None None] MFA Rules not processed for user `e96b136657864104a51900debd1fbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:09.964217 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-208de371-afe4-4b06-aad1-f14f231c2563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:10.046119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 657/1315] 10.222.0.65 () {58 vars in 1061 bytes} [Thu Jun 23 10:54:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:10.055090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb83ec4-7305-4aa6-b628-872cbbe8ece8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:10.056657 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8bb83ec4-7305-4aa6-b628-872cbbe8ece8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:10.167107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb83ec4-7305-4aa6-b628-872cbbe8ece8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:10.240819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:10.244115 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca4568-3e4b-4ab0-99fa-a323a5fff126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:10.244989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca4568-3e4b-4ab0-99fa-a323a5fff126 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:10.245394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca4568-3e4b-4ab0-99fa-a323a5fff126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:10.245808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca4568-3e4b-4ab0-99fa-a323a5fff126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:10.247778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e5ca4568-3e4b-4ab0-99fa-a323a5fff126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:10.357637 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca4568-3e4b-4ab0-99fa-a323a5fff126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:10.359013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca4568-3e4b-4ab0-99fa-a323a5fff126 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:10.377729 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 659/1316] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:54:12.919006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a36c20-87e2-444d-b42f-432f38e40b53 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:12.919566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a36c20-87e2-444d-b42f-432f38e40b53 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:12.919566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a36c20-87e2-444d-b42f-432f38e40b53 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:12.920953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 658/1317] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:12.926562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5c732-5cef-4958-9473-70b13c18e670 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:12.926967 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5c732-5cef-4958-9473-70b13c18e670 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:12.927112 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5c732-5cef-4958-9473-70b13c18e670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:12.989063 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fdb5c732-5cef-4958-9473-70b13c18e670 None None] MFA Rules not processed for user `95e96acf6a87449d9aa6f29de04b192f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:12.990257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fdb5c732-5cef-4958-9473-70b13c18e670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:13.040920 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:13.085412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 660/1318] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 10:54:13.125579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a36c20-87e2-444d-b42f-432f38e40b53 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:13.127115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-36a36c20-87e2-444d-b42f-432f38e40b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:13.228972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a36c20-87e2-444d-b42f-432f38e40b53 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:13.231781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:13.301637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca9a14-1a2a-4905-b25f-5cb4452f2925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:13.302284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca9a14-1a2a-4905-b25f-5cb4452f2925 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:13.302568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca9a14-1a2a-4905-b25f-5cb4452f2925 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:13.302845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca9a14-1a2a-4905-b25f-5cb4452f2925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:13.304529 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-43ca9a14-1a2a-4905-b25f-5cb4452f2925 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:13.409303 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca9a14-1a2a-4905-b25f-5cb4452f2925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:13.410722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca9a14-1a2a-4905-b25f-5cb4452f2925 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:13.428999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 659/1319] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:54:16.129270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb74366-6bb8-46cf-9d51-1766e01abd78 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:16.129718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb74366-6bb8-46cf-9d51-1766e01abd78 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:16.129815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb74366-6bb8-46cf-9d51-1766e01abd78 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:16.131188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 661/1320] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:16.136264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fadc5-f1df-4c41-b3a8-caedf13424e8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:16.136658 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fadc5-f1df-4c41-b3a8-caedf13424e8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:16.136823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fadc5-f1df-4c41-b3a8-caedf13424e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:16.175895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:16.201303 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-140fadc5-f1df-4c41-b3a8-caedf13424e8 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:16.202506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-140fadc5-f1df-4c41-b3a8-caedf13424e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:16.297086 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 660/1321] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:16.336213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb74366-6bb8-46cf-9d51-1766e01abd78 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:16.337741 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7cb74366-6bb8-46cf-9d51-1766e01abd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:16.438449 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb74366-6bb8-46cf-9d51-1766e01abd78 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:16.509765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f777ade-1a3b-4229-b7ea-9b27ece0cfc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:16.510400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f777ade-1a3b-4229-b7ea-9b27ece0cfc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:16.510683 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f777ade-1a3b-4229-b7ea-9b27ece0cfc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:16.510958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f777ade-1a3b-4229-b7ea-9b27ece0cfc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:16.512592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7f777ade-1a3b-4229-b7ea-9b27ece0cfc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:16.520358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:16.619805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f777ade-1a3b-4229-b7ea-9b27ece0cfc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:16.621230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f777ade-1a3b-4229-b7ea-9b27ece0cfc0 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:16.639575 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 662/1322] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:54:19.329181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb3385-8c6e-415b-b536-39b768c557da None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:19.329709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb3385-8c6e-415b-b536-39b768c557da None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:19.329709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb3385-8c6e-415b-b536-39b768c557da None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:19.331165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 661/1323] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:19.336447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db67e56-49ac-410f-85e7-e5e8512f534d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:19.336855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db67e56-49ac-410f-85e7-e5e8512f534d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:19.337121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db67e56-49ac-410f-85e7-e5e8512f534d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:19.400431 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7db67e56-49ac-410f-85e7-e5e8512f534d None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:19.401556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7db67e56-49ac-410f-85e7-e5e8512f534d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:19.460074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:19.503146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 663/1324] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:19.539451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb3385-8c6e-415b-b536-39b768c557da None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:19.541047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7ebb3385-8c6e-415b-b536-39b768c557da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:19.652623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb3385-8c6e-415b-b536-39b768c557da None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:19.665541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:19.726636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a562b-3f6c-4fc9-abd4-a686512a1c12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:19.727313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a562b-3f6c-4fc9-abd4-a686512a1c12 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:19.727593 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a562b-3f6c-4fc9-abd4-a686512a1c12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:19.727899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a562b-3f6c-4fc9-abd4-a686512a1c12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:19.729547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2d3a562b-3f6c-4fc9-abd4-a686512a1c12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:19.837311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a562b-3f6c-4fc9-abd4-a686512a1c12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:19.838754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a562b-3f6c-4fc9-abd4-a686512a1c12 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:19.856561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 662/1325] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:54:21.958195 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e73f7-cd51-4eb9-a41f-87ea362ae17c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:21.958709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e73f7-cd51-4eb9-a41f-87ea362ae17c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:21.958709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e73f7-cd51-4eb9-a41f-87ea362ae17c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:21.960082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 664/1326] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:21.965439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23e4f3-8236-463b-8e0a-1f0d51b4a2ac None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:21.965864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23e4f3-8236-463b-8e0a-1f0d51b4a2ac None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:21.965993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23e4f3-8236-463b-8e0a-1f0d51b4a2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:22.013662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:22.033706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-eb23e4f3-8236-463b-8e0a-1f0d51b4a2ac None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:22.034899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-eb23e4f3-8236-463b-8e0a-1f0d51b4a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:22.154538 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 663/1327] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:22.195451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e73f7-cd51-4eb9-a41f-87ea362ae17c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:22.197029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c14e73f7-cd51-4eb9-a41f-87ea362ae17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:22.318219 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e73f7-cd51-4eb9-a41f-87ea362ae17c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:22.397607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc6553-2f05-45f7-96c1-daa5c03516a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:22.398268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc6553-2f05-45f7-96c1-daa5c03516a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:22.398531 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc6553-2f05-45f7-96c1-daa5c03516a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:22.398802 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc6553-2f05-45f7-96c1-daa5c03516a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:22.400451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-08fc6553-2f05-45f7-96c1-daa5c03516a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:22.417575 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:22.508480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc6553-2f05-45f7-96c1-daa5c03516a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:22.509829 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc6553-2f05-45f7-96c1-daa5c03516a7 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:22.529735 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 665/1328] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:54:24.626440 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca70ab5-953b-4239-85c5-b7dafeb6d56c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:24.626975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca70ab5-953b-4239-85c5-b7dafeb6d56c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:24.626975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca70ab5-953b-4239-85c5-b7dafeb6d56c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:24.628406 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 664/1329] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:24.633839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76f645-c9f5-4447-b9fd-2ba3c07d60a3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:24.634285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76f645-c9f5-4447-b9fd-2ba3c07d60a3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:24.634376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76f645-c9f5-4447-b9fd-2ba3c07d60a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:24.701537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6e76f645-c9f5-4447-b9fd-2ba3c07d60a3 None None] MFA Rules not processed for user `95e96acf6a87449d9aa6f29de04b192f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:24.702755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6e76f645-c9f5-4447-b9fd-2ba3c07d60a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:24.770155 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:24.802719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 666/1330] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:24.839203 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca70ab5-953b-4239-85c5-b7dafeb6d56c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:24.840761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fca70ab5-953b-4239-85c5-b7dafeb6d56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:24.941976 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca70ab5-953b-4239-85c5-b7dafeb6d56c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:24.964519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:25.018719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf194b1-9f41-401a-9230-0851eeca45a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:25.019363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf194b1-9f41-401a-9230-0851eeca45a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:25.019640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf194b1-9f41-401a-9230-0851eeca45a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:25.019939 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf194b1-9f41-401a-9230-0851eeca45a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:25.021564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0bf194b1-9f41-401a-9230-0851eeca45a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:25.142028 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf194b1-9f41-401a-9230-0851eeca45a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:25.143812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf194b1-9f41-401a-9230-0851eeca45a7 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:25.162322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 665/1331] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:24 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:54:27.441808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f7f23-d8ea-4d9e-b2a0-bea5b4323c62 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:27.442293 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f7f23-d8ea-4d9e-b2a0-bea5b4323c62 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:27.442385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f7f23-d8ea-4d9e-b2a0-bea5b4323c62 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:27.443738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 667/1332] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:27.448840 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4bdea-b311-48ab-8d66-ee6c87c0e57d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:27.449287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4bdea-b311-48ab-8d66-ee6c87c0e57d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:27.449383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4bdea-b311-48ab-8d66-ee6c87c0e57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:27.512679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:27.516432 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-77e4bdea-b311-48ab-8d66-ee6c87c0e57d None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:27.518305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-77e4bdea-b311-48ab-8d66-ee6c87c0e57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:27.615782 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 666/1333] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:27.651071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f7f23-d8ea-4d9e-b2a0-bea5b4323c62 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:27.652624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-515f7f23-d8ea-4d9e-b2a0-bea5b4323c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:27.755104 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f7f23-d8ea-4d9e-b2a0-bea5b4323c62 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:27.825983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89733aee-f332-4a4c-b709-832d8335971d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:27.826626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89733aee-f332-4a4c-b709-832d8335971d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:27.826928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89733aee-f332-4a4c-b709-832d8335971d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:27.827188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89733aee-f332-4a4c-b709-832d8335971d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:27.828888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-89733aee-f332-4a4c-b709-832d8335971d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:27.855559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:27.935610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89733aee-f332-4a4c-b709-832d8335971d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:27.937000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89733aee-f332-4a4c-b709-832d8335971d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:27.955428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 668/1334] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:54:30.024625 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c142ba9-da41-45b6-8fd3-df7e2a971cb4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:30.025184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c142ba9-da41-45b6-8fd3-df7e2a971cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:30.025282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c142ba9-da41-45b6-8fd3-df7e2a971cb4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:30.026621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 667/1335] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:30.031942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce70bae-ccf3-4134-9715-9879e7a85b8d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:30.032346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce70bae-ccf3-4134-9715-9879e7a85b8d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:30.032505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce70bae-ccf3-4134-9715-9879e7a85b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:30.096556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6ce70bae-ccf3-4134-9715-9879e7a85b8d None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:30.097536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6ce70bae-ccf3-4134-9715-9879e7a85b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:30.169969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:30.193441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 669/1336] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:30.232391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c142ba9-da41-45b6-8fd3-df7e2a971cb4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:30.233904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7c142ba9-da41-45b6-8fd3-df7e2a971cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:30.340512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c142ba9-da41-45b6-8fd3-df7e2a971cb4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:30.368243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:30.412182 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe488726-57d5-4014-8cb8-b6d79a87d77e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:30.412829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe488726-57d5-4014-8cb8-b6d79a87d77e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:30.413113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe488726-57d5-4014-8cb8-b6d79a87d77e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:30.413391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe488726-57d5-4014-8cb8-b6d79a87d77e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:30.415017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fe488726-57d5-4014-8cb8-b6d79a87d77e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:30.520520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe488726-57d5-4014-8cb8-b6d79a87d77e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:30.521945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe488726-57d5-4014-8cb8-b6d79a87d77e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:30.539997 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 668/1337] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:54:32.863291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce95b3a-d423-4368-bff0-1224363ef031 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:32.863961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce95b3a-d423-4368-bff0-1224363ef031 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:32.863961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce95b3a-d423-4368-bff0-1224363ef031 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:32.865242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 670/1338] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:32.870743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5902d-ff17-4532-8b49-ef53bb6f4b2f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:32.871176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5902d-ff17-4532-8b49-ef53bb6f4b2f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:32.871282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5902d-ff17-4532-8b49-ef53bb6f4b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:32.934014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5da5902d-ff17-4532-8b49-ef53bb6f4b2f None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:32.935195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5da5902d-ff17-4532-8b49-ef53bb6f4b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:32.943632 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:33.030594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 669/1339] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:33.065662 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce95b3a-d423-4368-bff0-1224363ef031 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:33.067220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9ce95b3a-d423-4368-bff0-1224363ef031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:33.168582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce95b3a-d423-4368-bff0-1224363ef031 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:33.239470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfe255-dc5d-41cd-baad-0480ad9b1cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:33.240147 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfe255-dc5d-41cd-baad-0480ad9b1cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:33.240431 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfe255-dc5d-41cd-baad-0480ad9b1cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:33.240710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfe255-dc5d-41cd-baad-0480ad9b1cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:33.242330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-23bfe255-dc5d-41cd-baad-0480ad9b1cf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:33.276998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:33.351438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bfe255-dc5d-41cd-baad-0480ad9b1cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:33.352903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bfe255-dc5d-41cd-baad-0480ad9b1cf5 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:33.372049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 671/1340] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:54:37.407810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd09ec-b547-4011-9e93-017364a45a93 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:37.408314 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd09ec-b547-4011-9e93-017364a45a93 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:37.408314 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd09ec-b547-4011-9e93-017364a45a93 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:37.409664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 670/1341] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:37.414790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88f4ed-b82b-4c88-968e-df2c0e9b5aa0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:37.415198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88f4ed-b82b-4c88-968e-df2c0e9b5aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:37.415350 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88f4ed-b82b-4c88-968e-df2c0e9b5aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:37.486971 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4d88f4ed-b82b-4c88-968e-df2c0e9b5aa0 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:37.488200 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4d88f4ed-b82b-4c88-968e-df2c0e9b5aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:37.579833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:37.592883 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 672/1342] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:37.632799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd09ec-b547-4011-9e93-017364a45a93 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:37.635010 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bbdd09ec-b547-4011-9e93-017364a45a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:37.729014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd09ec-b547-4011-9e93-017364a45a93 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:37.732588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f460e-a1f5-42aa-b782-58644d1e4ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:37.733238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f460e-a1f5-42aa-b782-58644d1e4ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:37.733520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f460e-a1f5-42aa-b782-58644d1e4ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:37.733796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f460e-a1f5-42aa-b782-58644d1e4ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:37.743249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f460e-a1f5-42aa-b782-58644d1e4ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:37.744708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f460e-a1f5-42aa-b782-58644d1e4ec1 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:37.754859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 671/1343] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:54:40.377863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedb099-dabf-47f2-8302-bc41f46b458f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:40.378373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedb099-dabf-47f2-8302-bc41f46b458f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:40.378373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedb099-dabf-47f2-8302-bc41f46b458f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:40.379880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 673/1344] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:40.385060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bea39d-3822-4ba0-8113-29c0c7999609 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:40.385458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bea39d-3822-4ba0-8113-29c0c7999609 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:40.385626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bea39d-3822-4ba0-8113-29c0c7999609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:40.413486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-66bea39d-3822-4ba0-8113-29c0c7999609 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:40.414684 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-66bea39d-3822-4ba0-8113-29c0c7999609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:40.442396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:40.491809 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 672/1345] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:40.531421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedb099-dabf-47f2-8302-bc41f46b458f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:40.542428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedb099-dabf-47f2-8302-bc41f46b458f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:40.545992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf071a-c33c-444a-b642-6b32bf9ab440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:40.546641 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf071a-c33c-444a-b642-6b32bf9ab440 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:40.546921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf071a-c33c-444a-b642-6b32bf9ab440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:40.547194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf071a-c33c-444a-b642-6b32bf9ab440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:40.557027 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf071a-c33c-444a-b642-6b32bf9ab440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:40.558477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf071a-c33c-444a-b642-6b32bf9ab440 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:40.569673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 674/1346] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:54:44.527621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ef174-dba1-4fea-945e-425fc6e135be None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:44.528199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ef174-dba1-4fea-945e-425fc6e135be None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:44.528199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ef174-dba1-4fea-945e-425fc6e135be None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:44.529612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 673/1347] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:44.535045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8e6bb-da16-40f3-badd-4c229e18e52e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:44.535470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8e6bb-da16-40f3-badd-4c229e18e52e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:44.535600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8e6bb-da16-40f3-badd-4c229e18e52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:44.553372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-53e8e6bb-da16-40f3-badd-4c229e18e52e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:44.554580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-53e8e6bb-da16-40f3-badd-4c229e18e52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:44.562546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 675/1348] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:44.600761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ef174-dba1-4fea-945e-425fc6e135be None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:44.612035 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ef174-dba1-4fea-945e-425fc6e135be None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:44.615452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c07430-878f-4f90-ae18-f2ca2e416b06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:44.616121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c07430-878f-4f90-ae18-f2ca2e416b06 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:44.616405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c07430-878f-4f90-ae18-f2ca2e416b06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:44.616685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c07430-878f-4f90-ae18-f2ca2e416b06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:44.626481 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c07430-878f-4f90-ae18-f2ca2e416b06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:44.627955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c07430-878f-4f90-ae18-f2ca2e416b06 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:44.637600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 674/1349] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 10:54:49.707345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51415b-2e66-4d8a-bd89-f6bf668c0091 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:49.707847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51415b-2e66-4d8a-bd89-f6bf668c0091 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:49.708195 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51415b-2e66-4d8a-bd89-f6bf668c0091 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:49.709570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 676/1350] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 10:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:49.715901 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bf8b2-fbc9-4a6e-bcc8-6766dae43f09 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:49.716280 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bf8b2-fbc9-4a6e-bcc8-6766dae43f09 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:49.716379 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bf8b2-fbc9-4a6e-bcc8-6766dae43f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:49.801150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fe8bf8b2-fbc9-4a6e-bcc8-6766dae43f09 None None] MFA Rules not processed for user `18fc0e7956b14a49beab025922e8ff81`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:49.802345 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fe8bf8b2-fbc9-4a6e-bcc8-6766dae43f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:49.885288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 675/1351] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 10:54:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:54.181835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417716d-9b9c-41ba-97d8-d678d5dd9a20 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:54.182380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417716d-9b9c-41ba-97d8-d678d5dd9a20 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:54.182380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417716d-9b9c-41ba-97d8-d678d5dd9a20 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:54.183782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 677/1352] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 10:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:54.189682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295b4d4-b660-4be2-a6c4-58219c94c527 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:54.190187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295b4d4-b660-4be2-a6c4-58219c94c527 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:54.190301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295b4d4-b660-4be2-a6c4-58219c94c527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:54.209361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3295b4d4-b660-4be2-a6c4-58219c94c527 None None] MFA Rules not processed for user `18fc0e7956b14a49beab025922e8ff81`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:54.210828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3295b4d4-b660-4be2-a6c4-58219c94c527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:54.221321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 676/1353] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 10:54:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:54.548654 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=13 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:55.630645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7877e2-17a7-41d2-bada-4b099e51d883 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:55.631276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7877e2-17a7-41d2-bada-4b099e51d883 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:55.631276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7877e2-17a7-41d2-bada-4b099e51d883 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:55.632611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 678/1354] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 10:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:55.637561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe11726-0b7b-4fca-b2f0-0366a49684b1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:55.637990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe11726-0b7b-4fca-b2f0-0366a49684b1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:55.638107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe11726-0b7b-4fca-b2f0-0366a49684b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:55.655624 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2fe11726-0b7b-4fca-b2f0-0366a49684b1 None None] MFA Rules not processed for user `18fc0e7956b14a49beab025922e8ff81`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:55.656833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2fe11726-0b7b-4fca-b2f0-0366a49684b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:55.665504 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 677/1355] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 10:54:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 10:54:59.393745 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26600fb1-304e-4172-9a31-124e77723373 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:59.394276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26600fb1-304e-4172-9a31-124e77723373 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:59.394276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26600fb1-304e-4172-9a31-124e77723373 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:59.395546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 679/1356] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:54:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:59.401268 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151e20-49a2-4e71-9451-6723d8ae3f91 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:59.401687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151e20-49a2-4e71-9451-6723d8ae3f91 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:59.401828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151e20-49a2-4e71-9451-6723d8ae3f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:59.420348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-59151e20-49a2-4e71-9451-6723d8ae3f91 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:59.421553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-59151e20-49a2-4e71-9451-6723d8ae3f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:59.429643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 678/1357] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:54:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 10:54:59.488970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0b00f-83cc-42f6-8ff8-95f519f02aca None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:59.489694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0b00f-83cc-42f6-8ff8-95f519f02aca None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:59.490134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0b00f-83cc-42f6-8ff8-95f519f02aca None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:59.491637 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 680/1358] 10.222.0.65 () {54 vars in 961 bytes} [Thu Jun 23 10:54:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:54:59.500209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd029e4-eee7-48d2-87ea-e0f6b4fcb3a1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:59.500705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd029e4-eee7-48d2-87ea-e0f6b4fcb3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:59.501171 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd029e4-eee7-48d2-87ea-e0f6b4fcb3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:59.547437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:59.556412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3dd029e4-eee7-48d2-87ea-e0f6b4fcb3a1 None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:54:59.557612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3dd029e4-eee7-48d2-87ea-e0f6b4fcb3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:54:59.646755 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 679/1359] 10.222.0.65 () {58 vars in 1058 bytes} [Thu Jun 23 10:54:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:54:59.654489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0b00f-83cc-42f6-8ff8-95f519f02aca None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:54:59.670161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0b00f-83cc-42f6-8ff8-95f519f02aca None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:54:59.675425 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3c26d-f18a-49e9-a271-baf5b8c4ce15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:54:59.676180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3c26d-f18a-49e9-a271-baf5b8c4ce15 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:54:59.676457 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3c26d-f18a-49e9-a271-baf5b8c4ce15 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:54:59.676849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3c26d-f18a-49e9-a271-baf5b8c4ce15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:54:59.685727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3c26d-f18a-49e9-a271-baf5b8c4ce15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:54:59.687588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3c26d-f18a-49e9-a271-baf5b8c4ce15 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:54:59.698439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 681/1360] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:54:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:01.947499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06bd74-ba5f-42e9-bbe5-601b656daea3 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:01.948090 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06bd74-ba5f-42e9-bbe5-601b656daea3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:01.948189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06bd74-ba5f-42e9-bbe5-601b656daea3 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:01.949685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 680/1361] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:01.955218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c4459-eb3f-4fd5-8eee-6d62afdb3c51 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:01.955687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c4459-eb3f-4fd5-8eee-6d62afdb3c51 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:01.955800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c4459-eb3f-4fd5-8eee-6d62afdb3c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:01.973752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-840c4459-eb3f-4fd5-8eee-6d62afdb3c51 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:01.974957 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-840c4459-eb3f-4fd5-8eee-6d62afdb3c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:01.983013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 682/1362] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:02.027344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06bd74-ba5f-42e9-bbe5-601b656daea3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:02.042269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06bd74-ba5f-42e9-bbe5-601b656daea3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:02.047747 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb60e4-981f-468a-8312-f437471dbfbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:02.048498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb60e4-981f-468a-8312-f437471dbfbd service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:02.048786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb60e4-981f-468a-8312-f437471dbfbd service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:02.049069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb60e4-981f-468a-8312-f437471dbfbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:02.057433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb60e4-981f-468a-8312-f437471dbfbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:02.059149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb60e4-981f-468a-8312-f437471dbfbd service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:02.069408 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 681/1363] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:04.428221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2057d-6091-4a17-bce4-ef7042712e7c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:04.428740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2057d-6091-4a17-bce4-ef7042712e7c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:04.428845 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2057d-6091-4a17-bce4-ef7042712e7c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:04.430160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 683/1364] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:04.435297 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e3a83-5b81-4012-83fc-f7894a074c8e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:04.435760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e3a83-5b81-4012-83fc-f7894a074c8e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:04.435927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e3a83-5b81-4012-83fc-f7894a074c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:04.454497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a95e3a83-5b81-4012-83fc-f7894a074c8e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:04.456260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a95e3a83-5b81-4012-83fc-f7894a074c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:04.464905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 682/1365] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:04.515728 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577f2dd-5891-49b3-8809-f2a3ccbc5e23 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:04.516078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577f2dd-5891-49b3-8809-f2a3ccbc5e23 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:04.516307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577f2dd-5891-49b3-8809-f2a3ccbc5e23 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:04.518568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 684/1366] 10.222.0.65 () {54 vars in 961 bytes} [Thu Jun 23 10:55:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:04.524404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402370fb-48d0-4428-9f26-a6b06594a1ce None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:04.524700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402370fb-48d0-4428-9f26-a6b06594a1ce None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:04.524974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402370fb-48d0-4428-9f26-a6b06594a1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:04.549529 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-402370fb-48d0-4428-9f26-a6b06594a1ce None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:04.550735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-402370fb-48d0-4428-9f26-a6b06594a1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:04.560189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 683/1367] 10.222.0.65 () {58 vars in 1058 bytes} [Thu Jun 23 10:55:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:04.568587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577f2dd-5891-49b3-8809-f2a3ccbc5e23 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:04.583438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577f2dd-5891-49b3-8809-f2a3ccbc5e23 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:04.588825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bcc80b-1437-4579-a3ab-dfe6373e7901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:04.589465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcc80b-1437-4579-a3ab-dfe6373e7901 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:04.589747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcc80b-1437-4579-a3ab-dfe6373e7901 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:04.590024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcc80b-1437-4579-a3ab-dfe6373e7901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:04.598163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bcc80b-1437-4579-a3ab-dfe6373e7901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:04.599768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bcc80b-1437-4579-a3ab-dfe6373e7901 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:04.609404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 685/1368] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:06.680941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d519b0e-2462-4c58-bb35-2415c7b659af None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:06.681484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d519b0e-2462-4c58-bb35-2415c7b659af None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:06.681484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d519b0e-2462-4c58-bb35-2415c7b659af None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:06.682857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 684/1369] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:06.688394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6451a9-6e62-4bbe-9e2e-31d11ad02053 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:06.688825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6451a9-6e62-4bbe-9e2e-31d11ad02053 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:06.689134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6451a9-6e62-4bbe-9e2e-31d11ad02053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:06.709727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5f6451a9-6e62-4bbe-9e2e-31d11ad02053 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:06.711080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5f6451a9-6e62-4bbe-9e2e-31d11ad02053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:06.720966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 686/1370] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:06.765163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d519b0e-2462-4c58-bb35-2415c7b659af None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:06.779407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d519b0e-2462-4c58-bb35-2415c7b659af None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:06.785381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe7e43-6668-4933-ac16-e5ef00f2bdc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:06.786011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe7e43-6668-4933-ac16-e5ef00f2bdc3 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:06.786288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe7e43-6668-4933-ac16-e5ef00f2bdc3 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:06.786826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe7e43-6668-4933-ac16-e5ef00f2bdc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:06.795061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe7e43-6668-4933-ac16-e5ef00f2bdc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:06.796672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe7e43-6668-4933-ac16-e5ef00f2bdc3 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:06.806897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 685/1371] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:07.555564 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b62886-7c8a-48d9-944d-928eee262b8e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:07.555991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b62886-7c8a-48d9-944d-928eee262b8e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:07.556210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b62886-7c8a-48d9-944d-928eee262b8e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:07.557588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 687/1372] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:07.562796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbf369-5236-4266-9beb-9ffbb780669c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:07.563114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbf369-5236-4266-9beb-9ffbb780669c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:07.563418 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbf369-5236-4266-9beb-9ffbb780669c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:07.584201 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-84dbf369-5236-4266-9beb-9ffbb780669c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:07.585410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-84dbf369-5236-4266-9beb-9ffbb780669c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:07.593587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 686/1373] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:07.941861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3e6b7-ddb8-41e4-95e7-29cbbdf8576b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:07.941861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3e6b7-ddb8-41e4-95e7-29cbbdf8576b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:07.942584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3e6b7-ddb8-41e4-95e7-29cbbdf8576b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:07.943546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 688/1374] 10.222.0.65 () {54 vars in 963 bytes} [Thu Jun 23 10:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:07.948818 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccafdf-cbb6-4b92-82fe-7b260a4a612d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:07.949439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccafdf-cbb6-4b92-82fe-7b260a4a612d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:07.949439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccafdf-cbb6-4b92-82fe-7b260a4a612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:08.000376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-53ccafdf-cbb6-4b92-82fe-7b260a4a612d None None] MFA Rules not processed for user `3663612647934677b83fb42aaf06d6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:08.001638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-53ccafdf-cbb6-4b92-82fe-7b260a4a612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:08.068770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 687/1375] 10.222.0.65 () {58 vars in 1060 bytes} [Thu Jun 23 10:55:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:08.076126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3e6b7-ddb8-41e4-95e7-29cbbdf8576b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:08.087557 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3e6b7-ddb8-41e4-95e7-29cbbdf8576b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:08.091975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf137f-0968-4493-b014-4c25685f59ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:08.092869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf137f-0968-4493-b014-4c25685f59ea service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:08.093671 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf137f-0968-4493-b014-4c25685f59ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:08.094300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf137f-0968-4493-b014-4c25685f59ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:08.107544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf137f-0968-4493-b014-4c25685f59ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:08.109363 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf137f-0968-4493-b014-4c25685f59ea service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:08.121179 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 689/1376] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:08.868165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e510eb-31fb-4167-97bb-1b92dacdf59f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:08.868433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e510eb-31fb-4167-97bb-1b92dacdf59f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:08.869093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e510eb-31fb-4167-97bb-1b92dacdf59f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:08.874339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 688/1377] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:08.875608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e15279-5acd-49a6-b598-b0d351930aec None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:08.875757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e15279-5acd-49a6-b598-b0d351930aec None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:08.876148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e15279-5acd-49a6-b598-b0d351930aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:08.896146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-51e15279-5acd-49a6-b598-b0d351930aec None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:08.897717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-51e15279-5acd-49a6-b598-b0d351930aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:08.907545 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 690/1378] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:08.950923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e510eb-31fb-4167-97bb-1b92dacdf59f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:08.967928 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e510eb-31fb-4167-97bb-1b92dacdf59f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:08.974248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86409805-effa-4396-9176-373e0a38ced3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:08.975125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409805-effa-4396-9176-373e0a38ced3 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:08.975409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409805-effa-4396-9176-373e0a38ced3 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:08.975704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409805-effa-4396-9176-373e0a38ced3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:08.984869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86409805-effa-4396-9176-373e0a38ced3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:08.986499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86409805-effa-4396-9176-373e0a38ced3 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:08.990569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:55:08.997672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 689/1379] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:11.092426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0033-18cf-4679-b358-e283e3b27c49 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:11.093394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0033-18cf-4679-b358-e283e3b27c49 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:11.093878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0033-18cf-4679-b358-e283e3b27c49 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:11.095779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 691/1380] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:11.101196 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a2698-3eb1-4f65-b31e-8f5df82717fe None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:11.101968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a2698-3eb1-4f65-b31e-8f5df82717fe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:11.101968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a2698-3eb1-4f65-b31e-8f5df82717fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:11.126478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-af4a2698-3eb1-4f65-b31e-8f5df82717fe None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:11.128397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af4a2698-3eb1-4f65-b31e-8f5df82717fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:11.147835 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 690/1381] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 10:55:11.190704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a0033-18cf-4679-b358-e283e3b27c49 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:11.205687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a0033-18cf-4679-b358-e283e3b27c49 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:11.211051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5d6a7-6a6e-4c60-9138-740992c22b6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:11.211702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5d6a7-6a6e-4c60-9138-740992c22b6b service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:11.212065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5d6a7-6a6e-4c60-9138-740992c22b6b service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:11.212253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5d6a7-6a6e-4c60-9138-740992c22b6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:11.220861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5d6a7-6a6e-4c60-9138-740992c22b6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:11.222426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5d6a7-6a6e-4c60-9138-740992c22b6b service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:11.231636 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 692/1382] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:13.282061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93713f-56d4-4af0-af67-a2b04a79e14c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:13.282450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93713f-56d4-4af0-af67-a2b04a79e14c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:13.282616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93713f-56d4-4af0-af67-a2b04a79e14c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:13.284239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 691/1383] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:13.289412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa42aa5-7866-46a0-ac3b-feb84ceb9108 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:13.289796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa42aa5-7866-46a0-ac3b-feb84ceb9108 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:13.289968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa42aa5-7866-46a0-ac3b-feb84ceb9108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:13.307796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-efa42aa5-7866-46a0-ac3b-feb84ceb9108 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:13.308979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-efa42aa5-7866-46a0-ac3b-feb84ceb9108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:13.318193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 693/1384] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 10:55:13.359450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93713f-56d4-4af0-af67-a2b04a79e14c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:13.374169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93713f-56d4-4af0-af67-a2b04a79e14c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:13.380193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59331a70-43a8-4a4c-a846-f5c004c26d12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:13.380932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59331a70-43a8-4a4c-a846-f5c004c26d12 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:13.381318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59331a70-43a8-4a4c-a846-f5c004c26d12 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:13.381693 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59331a70-43a8-4a4c-a846-f5c004c26d12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:13.390082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59331a70-43a8-4a4c-a846-f5c004c26d12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:13.391844 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59331a70-43a8-4a4c-a846-f5c004c26d12 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:13.401975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 692/1385] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:15.435643 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03461a2-abf5-49f1-b1d0-0c57ef8fc319 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:15.436369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03461a2-abf5-49f1-b1d0-0c57ef8fc319 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:15.436369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03461a2-abf5-49f1-b1d0-0c57ef8fc319 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:15.443038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 694/1386] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:15.443944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b4f36-3a71-4584-a8a7-79b4f1f858db None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:15.444482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b4f36-3a71-4584-a8a7-79b4f1f858db None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:15.444482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b4f36-3a71-4584-a8a7-79b4f1f858db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:15.476626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d61b4f36-3a71-4584-a8a7-79b4f1f858db None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:15.477976 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d61b4f36-3a71-4584-a8a7-79b4f1f858db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:15.487794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 693/1387] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:15.530584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03461a2-abf5-49f1-b1d0-0c57ef8fc319 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:15.552731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03461a2-abf5-49f1-b1d0-0c57ef8fc319 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:15.561801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4f51a-25e6-423c-b6ca-a3c85734ca61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:15.562659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4f51a-25e6-423c-b6ca-a3c85734ca61 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:15.562975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4f51a-25e6-423c-b6ca-a3c85734ca61 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:15.563254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4f51a-25e6-423c-b6ca-a3c85734ca61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:15.572234 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4f51a-25e6-423c-b6ca-a3c85734ca61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:15.573843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4f51a-25e6-423c-b6ca-a3c85734ca61 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:15.583367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 695/1388] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:18.059617 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a712f-9735-4bf0-8583-516f52dc02ba None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:18.060323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a712f-9735-4bf0-8583-516f52dc02ba None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:18.060323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a712f-9735-4bf0-8583-516f52dc02ba None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:18.061637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 694/1389] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:18.067415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cd19d-fc22-46d5-bc5a-b62c7193079b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:18.067905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cd19d-fc22-46d5-bc5a-b62c7193079b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:18.068184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cd19d-fc22-46d5-bc5a-b62c7193079b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:18.092439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d27cd19d-fc22-46d5-bc5a-b62c7193079b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:18.094323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d27cd19d-fc22-46d5-bc5a-b62c7193079b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:18.104208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 696/1390] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:18.152916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a712f-9735-4bf0-8583-516f52dc02ba None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:18.168143 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a712f-9735-4bf0-8583-516f52dc02ba None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:18.173520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303bc18-ad6f-4b9c-b940-a4ca5bd818f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:18.174165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303bc18-ad6f-4b9c-b940-a4ca5bd818f9 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:18.174446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303bc18-ad6f-4b9c-b940-a4ca5bd818f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:18.174725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303bc18-ad6f-4b9c-b940-a4ca5bd818f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:18.183080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303bc18-ad6f-4b9c-b940-a4ca5bd818f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:18.184715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303bc18-ad6f-4b9c-b940-a4ca5bd818f9 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:18.194119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 695/1391] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:20.271018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcef5d-e5bc-4ce7-9a85-c8a9c50103ee None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:20.271554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcef5d-e5bc-4ce7-9a85-c8a9c50103ee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:20.271824 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcef5d-e5bc-4ce7-9a85-c8a9c50103ee None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:20.273284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 697/1392] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:20.282216 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4c5f3-6198-44ca-ab71-0340bb6928fc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:20.282678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4c5f3-6198-44ca-ab71-0340bb6928fc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:20.282771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4c5f3-6198-44ca-ab71-0340bb6928fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:20.303789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-30b4c5f3-6198-44ca-ab71-0340bb6928fc None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:20.305812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-30b4c5f3-6198-44ca-ab71-0340bb6928fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:20.316318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 696/1393] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:20.360747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcef5d-e5bc-4ce7-9a85-c8a9c50103ee None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:20.376279 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcef5d-e5bc-4ce7-9a85-c8a9c50103ee None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:20.383485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef4331-fe99-4049-a357-ab60cd3113f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:20.384583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef4331-fe99-4049-a357-ab60cd3113f1 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:20.385098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef4331-fe99-4049-a357-ab60cd3113f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:20.385526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef4331-fe99-4049-a357-ab60cd3113f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:20.400226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef4331-fe99-4049-a357-ab60cd3113f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:20.402906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef4331-fe99-4049-a357-ab60cd3113f1 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:20.419843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 698/1394] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:22.480283 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41a4eb-a752-44a1-b47d-752c123abbc5 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:22.480811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41a4eb-a752-44a1-b47d-752c123abbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:22.480811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41a4eb-a752-44a1-b47d-752c123abbc5 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:22.482262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 697/1395] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:22.487523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6140352-c810-49cd-89f5-bf8167dddb57 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:22.487938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6140352-c810-49cd-89f5-bf8167dddb57 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:22.488122 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6140352-c810-49cd-89f5-bf8167dddb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:22.506467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d6140352-c810-49cd-89f5-bf8167dddb57 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:22.507689 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d6140352-c810-49cd-89f5-bf8167dddb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:22.515768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 699/1396] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:22.559200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41a4eb-a752-44a1-b47d-752c123abbc5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:22.573732 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41a4eb-a752-44a1-b47d-752c123abbc5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:22.579070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da59505-89e3-4b7a-a6f9-d823ee3f3128 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:22.579725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da59505-89e3-4b7a-a6f9-d823ee3f3128 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:22.580092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da59505-89e3-4b7a-a6f9-d823ee3f3128 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:22.580884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da59505-89e3-4b7a-a6f9-d823ee3f3128 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:22.588955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da59505-89e3-4b7a-a6f9-d823ee3f3128 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:22.590536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da59505-89e3-4b7a-a6f9-d823ee3f3128 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:22.600365 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 698/1397] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:23.363149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb5cb2-d057-4f30-9257-3e4cc7cc26e1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:23.363460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb5cb2-d057-4f30-9257-3e4cc7cc26e1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:23.363801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb5cb2-d057-4f30-9257-3e4cc7cc26e1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:23.365183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 700/1398] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:23.370385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b319d1-563f-4eff-ac0e-5900633efbf6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:23.370671 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b319d1-563f-4eff-ac0e-5900633efbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:23.370946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b319d1-563f-4eff-ac0e-5900633efbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:23.388593 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d5b319d1-563f-4eff-ac0e-5900633efbf6 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:23.389774 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d5b319d1-563f-4eff-ac0e-5900633efbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:23.398093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 699/1399] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:23.453301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20065d93-ecfa-49d7-94b3-50629b6b2ed1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:23.453593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20065d93-ecfa-49d7-94b3-50629b6b2ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:23.453872 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20065d93-ecfa-49d7-94b3-50629b6b2ed1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:23.455806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 701/1400] 10.222.0.65 () {54 vars in 962 bytes} [Thu Jun 23 10:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:23.461008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1bd07-4ad5-4ff4-8d17-51fd6aaf363e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:23.461301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1bd07-4ad5-4ff4-8d17-51fd6aaf363e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:23.461585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1bd07-4ad5-4ff4-8d17-51fd6aaf363e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:23.509403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-20f1bd07-4ad5-4ff4-8d17-51fd6aaf363e None None] MFA Rules not processed for user `5af6cc8c56db48beae2550954446c119`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:23.510607 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-20f1bd07-4ad5-4ff4-8d17-51fd6aaf363e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:23.569033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 700/1401] 10.222.0.65 () {58 vars in 1059 bytes} [Thu Jun 23 10:55:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:23.576506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20065d93-ecfa-49d7-94b3-50629b6b2ed1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:23.587405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20065d93-ecfa-49d7-94b3-50629b6b2ed1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:23.590860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab5a87-2b01-4d96-a30b-ced4977b1e65 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af6cc8c56db48beae2550954446c119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:23.591490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab5a87-2b01-4d96-a30b-ced4977b1e65 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:23.591810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab5a87-2b01-4d96-a30b-ced4977b1e65 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:23.592087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab5a87-2b01-4d96-a30b-ced4977b1e65 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:23.601499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab5a87-2b01-4d96-a30b-ced4977b1e65 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:23.602914 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab5a87-2b01-4d96-a30b-ced4977b1e65 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:23.612325 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 702/1402] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:24.317071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ec04e-785e-44e5-b1b6-eb54b8149fe4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:24.317417 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ec04e-785e-44e5-b1b6-eb54b8149fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:24.317640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ec04e-785e-44e5-b1b6-eb54b8149fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:24.367744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4f9ec04e-785e-44e5-b1b6-eb54b8149fe4 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:24.368955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4f9ec04e-785e-44e5-b1b6-eb54b8149fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:24.410220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:55:24.433788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 701/1403] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 10:55:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:24.479973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f49c7-7ebb-4039-b245-cc74f1a418d2 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:24.480264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f49c7-7ebb-4039-b245-cc74f1a418d2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:24.480545 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f49c7-7ebb-4039-b245-cc74f1a418d2 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:24.481959 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 703/1404] 10.222.0.65 () {54 vars in 942 bytes} [Thu Jun 23 10:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:24.488514 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc3ed3-fe85-4464-be99-622d903b17f8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:24.488808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc3ed3-fe85-4464-be99-622d903b17f8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:24.489085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc3ed3-fe85-4464-be99-622d903b17f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:24.537377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-26dc3ed3-fe85-4464-be99-622d903b17f8 None None] MFA Rules not processed for user `d3d8f8acd0484857b451877baf03e895`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:24.538582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-26dc3ed3-fe85-4464-be99-622d903b17f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:24.608746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 702/1405] 10.222.0.65 () {58 vars in 1039 bytes} [Thu Jun 23 10:55:24 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:24.617939 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f49c7-7ebb-4039-b245-cc74f1a418d2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:24.625507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=61 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:55:24.633147 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f49c7-7ebb-4039-b245-cc74f1a418d2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:24.638221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cca4e-0275-4833-8699-696cecd72ced service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:24.638861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cca4e-0275-4833-8699-696cecd72ced service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:24.639139 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cca4e-0275-4833-8699-696cecd72ced service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:24.639415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cca4e-0275-4833-8699-696cecd72ced service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:24.648702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cca4e-0275-4833-8699-696cecd72ced service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:24.650276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cca4e-0275-4833-8699-696cecd72ced service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:24.654350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b5f09-a4d9-4647-b678-7cab8e2b813e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:24.654649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b5f09-a4d9-4647-b678-7cab8e2b813e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:24.654961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b5f09-a4d9-4647-b678-7cab8e2b813e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:24.656584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 703/1406] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:55:24.659269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 704/1407] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 10:55:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 10:55:24.661618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462de346-bf30-49b3-a323-f03ede00f10b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:24.661830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462de346-bf30-49b3-a323-f03ede00f10b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:24.662101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462de346-bf30-49b3-a323-f03ede00f10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:24.680189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-462de346-bf30-49b3-a323-f03ede00f10b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:24.681417 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-462de346-bf30-49b3-a323-f03ede00f10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:24.690037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 704/1408] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:55:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 10:55:24.738414 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cca4e-0275-4833-8699-696cecd72ced service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:24.760790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cca4e-0275-4833-8699-696cecd72ced service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:24.767640 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ae64f-b3e0-4e6b-a2eb-f02cbff9130d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:24.768567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ae64f-b3e0-4e6b-a2eb-f02cbff9130d service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:24.768915 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ae64f-b3e0-4e6b-a2eb-f02cbff9130d service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:24.769260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ae64f-b3e0-4e6b-a2eb-f02cbff9130d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:24.778715 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ae64f-b3e0-4e6b-a2eb-f02cbff9130d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:24.780406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ae64f-b3e0-4e6b-a2eb-f02cbff9130d service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:24.794136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 705/1409] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:34.675093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=56 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:55:34.787166 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=11 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:55:46.762934 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54776787-fd94-407e-8e28-fa80260faec5 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:46.763481 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54776787-fd94-407e-8e28-fa80260faec5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:46.763481 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54776787-fd94-407e-8e28-fa80260faec5 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:46.764884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 705/1410] 10.222.0.65 () {54 vars in 918 bytes} [Thu Jun 23 10:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:46.770031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d24633-6146-4ffc-a9c4-e111a9795a56 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:46.770458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d24633-6146-4ffc-a9c4-e111a9795a56 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:46.770593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d24633-6146-4ffc-a9c4-e111a9795a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:46.788104 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-79d24633-6146-4ffc-a9c4-e111a9795a56 None None] MFA Rules not processed for user `18fc0e7956b14a49beab025922e8ff81`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:46.789421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-79d24633-6146-4ffc-a9c4-e111a9795a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:46.798390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 706/1411] 10.222.0.65 () {58 vars in 1015 bytes} [Thu Jun 23 10:55:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:46.812746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff346d-b656-44e1-a08b-244763e342a7 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:46.813034 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff346d-b656-44e1-a08b-244763e342a7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:46.813314 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff346d-b656-44e1-a08b-244763e342a7 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:46.814649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 706/1412] 10.222.0.65 () {54 vars in 918 bytes} [Thu Jun 23 10:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:46.819645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88117111-2980-4ee7-98f2-cd5e8b3786ff None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:46.819954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88117111-2980-4ee7-98f2-cd5e8b3786ff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:46.820255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88117111-2980-4ee7-98f2-cd5e8b3786ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:46.837405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-88117111-2980-4ee7-98f2-cd5e8b3786ff None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:46.838569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-88117111-2980-4ee7-98f2-cd5e8b3786ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:46.847682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 707/1413] 10.222.0.65 () {58 vars in 1015 bytes} [Thu Jun 23 10:55:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:49.313863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def045de-0ef0-432b-b1b5-43fc8c3d8bb9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:49.314437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def045de-0ef0-432b-b1b5-43fc8c3d8bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:49.314550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def045de-0ef0-432b-b1b5-43fc8c3d8bb9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:49.315947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 10:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:49.321029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8801fe-9c30-45c6-a8ef-4e24005be06c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:49.321435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8801fe-9c30-45c6-a8ef-4e24005be06c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:49.321591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8801fe-9c30-45c6-a8ef-4e24005be06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:49.340076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4e8801fe-9c30-45c6-a8ef-4e24005be06c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:49.341279 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4e8801fe-9c30-45c6-a8ef-4e24005be06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:49.349678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 10:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:49.393675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def045de-0ef0-432b-b1b5-43fc8c3d8bb9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:49.408287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def045de-0ef0-432b-b1b5-43fc8c3d8bb9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:49.414175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ff30b-c1bd-4ccf-8b95-7fb4214863bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:49.414807 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ff30b-c1bd-4ccf-8b95-7fb4214863bf service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:49.415091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ff30b-c1bd-4ccf-8b95-7fb4214863bf service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:49.415369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ff30b-c1bd-4ccf-8b95-7fb4214863bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:49.423530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ff30b-c1bd-4ccf-8b95-7fb4214863bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:49.425756 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ff30b-c1bd-4ccf-8b95-7fb4214863bf service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:49.435050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 708/1416] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 10:55:50.324419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934c7d2-b389-4393-a6b7-a456a02d7d9a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:50.324823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934c7d2-b389-4393-a6b7-a456a02d7d9a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:50.324979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934c7d2-b389-4393-a6b7-a456a02d7d9a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:50.326387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 10:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:50.331221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f5135-ffaf-4e40-b0a7-7aa56a3c3e56 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:50.331622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f5135-ffaf-4e40-b0a7-7aa56a3c3e56 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:50.331814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f5135-ffaf-4e40-b0a7-7aa56a3c3e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:50.349958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-782f5135-ffaf-4e40-b0a7-7aa56a3c3e56 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:50.351091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-782f5135-ffaf-4e40-b0a7-7aa56a3c3e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:50.358779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 10:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:50.508313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934c7d2-b389-4393-a6b7-a456a02d7d9a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:50.522807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934c7d2-b389-4393-a6b7-a456a02d7d9a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:50.528157 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c01c62-53ba-44c7-8fda-82847e60cf6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:50.528782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c01c62-53ba-44c7-8fda-82847e60cf6e service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:50.529065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c01c62-53ba-44c7-8fda-82847e60cf6e service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:50.529347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c01c62-53ba-44c7-8fda-82847e60cf6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:50.537797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c01c62-53ba-44c7-8fda-82847e60cf6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:50.539358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c01c62-53ba-44c7-8fda-82847e60cf6e service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:50.548578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 710/1419] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:51.414392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56162f37-5be6-4988-9f02-ddc7bc46db06 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:51.414923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56162f37-5be6-4988-9f02-ddc7bc46db06 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:51.414923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56162f37-5be6-4988-9f02-ddc7bc46db06 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:51.416340 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 10:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:51.421163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2cf49-7f34-4cc9-9a6a-5d4585751866 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:51.421590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2cf49-7f34-4cc9-9a6a-5d4585751866 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:51.421736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2cf49-7f34-4cc9-9a6a-5d4585751866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:51.439373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cfb2cf49-7f34-4cc9-9a6a-5d4585751866 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:51.440604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cfb2cf49-7f34-4cc9-9a6a-5d4585751866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:51.448441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 10:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:51.599557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56162f37-5be6-4988-9f02-ddc7bc46db06 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:51.613929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56162f37-5be6-4988-9f02-ddc7bc46db06 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:51.619380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11204-7150-4776-ae6e-f15a1b4b1514 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:51.620052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11204-7150-4776-ae6e-f15a1b4b1514 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:51.620579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11204-7150-4776-ae6e-f15a1b4b1514 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:51.620882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11204-7150-4776-ae6e-f15a1b4b1514 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:51.629754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11204-7150-4776-ae6e-f15a1b4b1514 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:51.631354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11204-7150-4776-ae6e-f15a1b4b1514 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:51.640816 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 711/1422] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:52.497519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44c930-5c7c-47ce-a076-698fa2cebf73 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:52.498034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44c930-5c7c-47ce-a076-698fa2cebf73 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:52.498034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44c930-5c7c-47ce-a076-698fa2cebf73 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:52.499374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 10:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:52.505115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac629e-ba87-48b1-be85-eee90da6ade5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:52.505547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac629e-ba87-48b1-be85-eee90da6ade5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:52.505710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac629e-ba87-48b1-be85-eee90da6ade5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:52.524338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-82ac629e-ba87-48b1-be85-eee90da6ade5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:52.525535 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-82ac629e-ba87-48b1-be85-eee90da6ade5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:52.533316 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 10:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:52.683467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44c930-5c7c-47ce-a076-698fa2cebf73 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:52.698245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44c930-5c7c-47ce-a076-698fa2cebf73 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:52.703608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d27b39-5342-4768-a8bb-2d3c0ed4b111 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:52.704263 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d27b39-5342-4768-a8bb-2d3c0ed4b111 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:52.704548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d27b39-5342-4768-a8bb-2d3c0ed4b111 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:52.704824 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d27b39-5342-4768-a8bb-2d3c0ed4b111 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:52.713078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d27b39-5342-4768-a8bb-2d3c0ed4b111 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:52.714632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d27b39-5342-4768-a8bb-2d3c0ed4b111 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:52.723892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 713/1425] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:53.572472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962f86d-7e70-4984-a3b4-62b4bb17a98d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:53.573013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962f86d-7e70-4984-a3b4-62b4bb17a98d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:53.573013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962f86d-7e70-4984-a3b4-62b4bb17a98d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:53.574403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 10:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:53.579320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a7e96-36dc-4ae6-a19d-2f5a5eae496c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:53.579753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a7e96-36dc-4ae6-a19d-2f5a5eae496c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:53.579907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a7e96-36dc-4ae6-a19d-2f5a5eae496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:53.598659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-689a7e96-36dc-4ae6-a19d-2f5a5eae496c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:53.599882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-689a7e96-36dc-4ae6-a19d-2f5a5eae496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:53.607631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 10:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:53.757209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962f86d-7e70-4984-a3b4-62b4bb17a98d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:53.772185 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962f86d-7e70-4984-a3b4-62b4bb17a98d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:53.783888 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1dfc1-b73b-4406-a056-f9b402cc4622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:53.784587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1dfc1-b73b-4406-a056-f9b402cc4622 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:53.784871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1dfc1-b73b-4406-a056-f9b402cc4622 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:53.785150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1dfc1-b73b-4406-a056-f9b402cc4622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:53.793543 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1dfc1-b73b-4406-a056-f9b402cc4622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:53.795124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1dfc1-b73b-4406-a056-f9b402cc4622 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:53.804760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 714/1428] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:54.602375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba37909-af4a-4ee8-83fb-2e2dba54d9ca None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:54.602904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba37909-af4a-4ee8-83fb-2e2dba54d9ca None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:54.603019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba37909-af4a-4ee8-83fb-2e2dba54d9ca None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:54.604325 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 10:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:54.609161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db8d7c-adab-41da-9f4a-02002caece97 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:54.609616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db8d7c-adab-41da-9f4a-02002caece97 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:54.609721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db8d7c-adab-41da-9f4a-02002caece97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:54.627746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-81db8d7c-adab-41da-9f4a-02002caece97 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:54.629761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-81db8d7c-adab-41da-9f4a-02002caece97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:54.637464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 10:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 10:55:54.785512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba37909-af4a-4ee8-83fb-2e2dba54d9ca None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:54.800382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba37909-af4a-4ee8-83fb-2e2dba54d9ca None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:54.805797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb4491-a70e-4922-944c-5c0b34c82fd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:54.806429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb4491-a70e-4922-944c-5c0b34c82fd6 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:54.806712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb4491-a70e-4922-944c-5c0b34c82fd6 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:54.806991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb4491-a70e-4922-944c-5c0b34c82fd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:54.815012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb4491-a70e-4922-944c-5c0b34c82fd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:54.816610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb4491-a70e-4922-944c-5c0b34c82fd6 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:54.825891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 716/1431] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:55.624291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f1693-0b63-4196-b8af-89bdfc78b935 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:55.624880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f1693-0b63-4196-b8af-89bdfc78b935 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:55.624880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f1693-0b63-4196-b8af-89bdfc78b935 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:55.626553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 10:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:55.632083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77624e54-9daa-4a9b-beaf-335c186f0447 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:55.632521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77624e54-9daa-4a9b-beaf-335c186f0447 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:55.632626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77624e54-9daa-4a9b-beaf-335c186f0447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:55.650470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-77624e54-9daa-4a9b-beaf-335c186f0447 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:55.651678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-77624e54-9daa-4a9b-beaf-335c186f0447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:55.659522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 10:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:55.807028 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f1693-0b63-4196-b8af-89bdfc78b935 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:55.821575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f1693-0b63-4196-b8af-89bdfc78b935 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:55.826805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3de922-abdb-4aaa-a88e-a15398b7d410 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:55.827453 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3de922-abdb-4aaa-a88e-a15398b7d410 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:55.827770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3de922-abdb-4aaa-a88e-a15398b7d410 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:55.828047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3de922-abdb-4aaa-a88e-a15398b7d410 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:55.835977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3de922-abdb-4aaa-a88e-a15398b7d410 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:55.837543 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3de922-abdb-4aaa-a88e-a15398b7d410 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:55.846634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 717/1434] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:56.641961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e2e7d-5dbb-4dc5-8753-00508204c60f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:56.642499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e2e7d-5dbb-4dc5-8753-00508204c60f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:56.642499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e2e7d-5dbb-4dc5-8753-00508204c60f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:56.644453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 10:55:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:56.649450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743d5ac-84dc-42eb-8c2a-cec93a77bbc4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:56.649874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743d5ac-84dc-42eb-8c2a-cec93a77bbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:56.649989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743d5ac-84dc-42eb-8c2a-cec93a77bbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:56.668615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7743d5ac-84dc-42eb-8c2a-cec93a77bbc4 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:56.669832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7743d5ac-84dc-42eb-8c2a-cec93a77bbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:56.677811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:56.826188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e2e7d-5dbb-4dc5-8753-00508204c60f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:56.840921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e2e7d-5dbb-4dc5-8753-00508204c60f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:56.846250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ade44-ec0c-4760-9285-b213ff5b1c28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:56.846908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ade44-ec0c-4760-9285-b213ff5b1c28 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:56.847192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ade44-ec0c-4760-9285-b213ff5b1c28 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:56.847471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ade44-ec0c-4760-9285-b213ff5b1c28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:56.855480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ade44-ec0c-4760-9285-b213ff5b1c28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:56.857068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ade44-ec0c-4760-9285-b213ff5b1c28 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:56.866351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 719/1437] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:57.660668 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deef85-acf2-4e3b-b013-08921f702d94 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:57.661236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deef85-acf2-4e3b-b013-08921f702d94 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:57.661236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deef85-acf2-4e3b-b013-08921f702d94 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:57.662615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 10:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:57.667440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b99259-3d21-4397-ad13-c6b8cc2242b3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:57.667876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b99259-3d21-4397-ad13-c6b8cc2242b3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:57.668047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b99259-3d21-4397-ad13-c6b8cc2242b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:57.686217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-02b99259-3d21-4397-ad13-c6b8cc2242b3 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:57.687473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-02b99259-3d21-4397-ad13-c6b8cc2242b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:57.695215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:57.843847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deef85-acf2-4e3b-b013-08921f702d94 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:57.858371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deef85-acf2-4e3b-b013-08921f702d94 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:57.864518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38922282-982a-404c-ac3c-1a8e5fa85f48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:57.865149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38922282-982a-404c-ac3c-1a8e5fa85f48 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:57.865431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38922282-982a-404c-ac3c-1a8e5fa85f48 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:57.865704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38922282-982a-404c-ac3c-1a8e5fa85f48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:57.873782 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38922282-982a-404c-ac3c-1a8e5fa85f48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:57.875341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38922282-982a-404c-ac3c-1a8e5fa85f48 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:57.884913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 720/1440] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:55:59.799874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d6365-079a-4d29-a097-5b8ee5a303ee None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:59.800508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d6365-079a-4d29-a097-5b8ee5a303ee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:59.800899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d6365-079a-4d29-a097-5b8ee5a303ee None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:59.803477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 10:55:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:55:59.808896 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce68bcd-482c-4bd7-b488-357ffabe51cd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:59.809706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce68bcd-482c-4bd7-b488-357ffabe51cd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:59.809989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce68bcd-482c-4bd7-b488-357ffabe51cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:59.828135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2ce68bcd-482c-4bd7-b488-357ffabe51cd None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:55:59.829313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2ce68bcd-482c-4bd7-b488-357ffabe51cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:55:59.837653 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 10:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:55:59.874652 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d6365-079a-4d29-a097-5b8ee5a303ee None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:55:59.886035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d6365-079a-4d29-a097-5b8ee5a303ee None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:55:59.889536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d35d88-6a8a-4478-8640-60b698d4523b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:55:59.890170 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d35d88-6a8a-4478-8640-60b698d4523b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:55:59.890451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d35d88-6a8a-4478-8640-60b698d4523b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:55:59.890751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d35d88-6a8a-4478-8640-60b698d4523b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:55:59.900861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d35d88-6a8a-4478-8640-60b698d4523b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:55:59.902278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d35d88-6a8a-4478-8640-60b698d4523b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:55:59.911533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 722/1443] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:02.134955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fd75a-62e9-414a-855a-856b82d1b93a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:02.135451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fd75a-62e9-414a-855a-856b82d1b93a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:02.135562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fd75a-62e9-414a-855a-856b82d1b93a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:02.136963 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 10:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:02.142033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd037-7a2b-4c9a-9254-6cfd477fb516 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:02.142425 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd037-7a2b-4c9a-9254-6cfd477fb516 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:02.142588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd037-7a2b-4c9a-9254-6cfd477fb516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:02.162561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ed4cd037-7a2b-4c9a-9254-6cfd477fb516 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:02.163802 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ed4cd037-7a2b-4c9a-9254-6cfd477fb516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:02.171643 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 10:56:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:02.209541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fd75a-62e9-414a-855a-856b82d1b93a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:02.220139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fd75a-62e9-414a-855a-856b82d1b93a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:02.224167 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c40bb-6515-45c6-8f40-3a095c97d02f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:02.224799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c40bb-6515-45c6-8f40-3a095c97d02f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:02.225082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c40bb-6515-45c6-8f40-3a095c97d02f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:02.225364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c40bb-6515-45c6-8f40-3a095c97d02f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:02.234150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c40bb-6515-45c6-8f40-3a095c97d02f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:02.235576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c40bb-6515-45c6-8f40-3a095c97d02f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:02.245475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 723/1446] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:04.543660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76189be-e3e1-4000-8d57-e049fc31ed8d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:04.545252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76189be-e3e1-4000-8d57-e049fc31ed8d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:04.545252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76189be-e3e1-4000-8d57-e049fc31ed8d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:04.545606 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 10:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:04.550789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64a52b-70c4-4c81-82f3-8daaed106ab7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:04.551231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64a52b-70c4-4c81-82f3-8daaed106ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:04.551361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64a52b-70c4-4c81-82f3-8daaed106ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:04.569193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8c64a52b-70c4-4c81-82f3-8daaed106ab7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:04.570376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8c64a52b-70c4-4c81-82f3-8daaed106ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:04.578018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 10:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:04.616503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76189be-e3e1-4000-8d57-e049fc31ed8d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:04.628047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76189be-e3e1-4000-8d57-e049fc31ed8d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:04.631556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac53e89-fad5-44a7-8264-20df5da82b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:04.632215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53e89-fad5-44a7-8264-20df5da82b8f service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:04.632502 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53e89-fad5-44a7-8264-20df5da82b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:04.632782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53e89-fad5-44a7-8264-20df5da82b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:04.641624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac53e89-fad5-44a7-8264-20df5da82b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:04.643054 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac53e89-fad5-44a7-8264-20df5da82b8f service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:04.652335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 725/1449] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:06.782477 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254a012-4fe9-439b-bc87-b489b0a0eaa3 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:06.783511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254a012-4fe9-439b-bc87-b489b0a0eaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:06.783623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254a012-4fe9-439b-bc87-b489b0a0eaa3 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:06.785012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 10:56:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:06.790321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174f125-fa8b-4c18-a4f6-c2012cac6c88 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:06.790754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174f125-fa8b-4c18-a4f6-c2012cac6c88 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:06.790940 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174f125-fa8b-4c18-a4f6-c2012cac6c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:06.808711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7174f125-fa8b-4c18-a4f6-c2012cac6c88 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:06.809896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7174f125-fa8b-4c18-a4f6-c2012cac6c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:06.817639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 10:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:06.856117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254a012-4fe9-439b-bc87-b489b0a0eaa3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:06.867275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254a012-4fe9-439b-bc87-b489b0a0eaa3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:06.870811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf975d-5112-42f8-afcd-9737ec8060ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:06.871438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf975d-5112-42f8-afcd-9737ec8060ff service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:06.871753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf975d-5112-42f8-afcd-9737ec8060ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:06.872033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf975d-5112-42f8-afcd-9737ec8060ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:06.881071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf975d-5112-42f8-afcd-9737ec8060ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:06.882440 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf975d-5112-42f8-afcd-9737ec8060ff service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:06.891597 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 726/1452] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:08.943564 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7d725-c6c4-48db-9f4d-ff731fc71887 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:08.944154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7d725-c6c4-48db-9f4d-ff731fc71887 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:08.944154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7d725-c6c4-48db-9f4d-ff731fc71887 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:08.945457 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 10:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:56:08.950789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7648475-8352-480b-95de-e36273b56075 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:08.951200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7648475-8352-480b-95de-e36273b56075 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:08.951447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7648475-8352-480b-95de-e36273b56075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:08.970268 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d7648475-8352-480b-95de-e36273b56075 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:08.971456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d7648475-8352-480b-95de-e36273b56075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:08.979027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 10:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:09.017961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7d725-c6c4-48db-9f4d-ff731fc71887 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:09.019642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b3b7d725-c6c4-48db-9f4d-ff731fc71887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:09.032218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7d725-c6c4-48db-9f4d-ff731fc71887 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:09.035678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097cc4a-a6dd-46f2-a69e-09b0654e5bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:09.036307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097cc4a-a6dd-46f2-a69e-09b0654e5bce service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:09.036587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097cc4a-a6dd-46f2-a69e-09b0654e5bce service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:09.036862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097cc4a-a6dd-46f2-a69e-09b0654e5bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:09.045469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097cc4a-a6dd-46f2-a69e-09b0654e5bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:09.046874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097cc4a-a6dd-46f2-a69e-09b0654e5bce service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:09.056172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 728/1455] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:11.162768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9721849-2720-42be-b5c4-adc8fc39726e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:11.163311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9721849-2720-42be-b5c4-adc8fc39726e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:11.163311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9721849-2720-42be-b5c4-adc8fc39726e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:11.164793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 10:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:11.169981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f171a-25fa-418e-8922-d02636e59a80 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:11.170406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f171a-25fa-418e-8922-d02636e59a80 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:11.170541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f171a-25fa-418e-8922-d02636e59a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:11.188525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f96f171a-25fa-418e-8922-d02636e59a80 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:11.189709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f96f171a-25fa-418e-8922-d02636e59a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:11.197369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 10:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:11.234847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9721849-2720-42be-b5c4-adc8fc39726e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:11.246252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9721849-2720-42be-b5c4-adc8fc39726e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:11.250032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d09e4-92cd-4363-a2f7-2eb7d0213f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:11.250661 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d09e4-92cd-4363-a2f7-2eb7d0213f25 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:11.250951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d09e4-92cd-4363-a2f7-2eb7d0213f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:11.251227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d09e4-92cd-4363-a2f7-2eb7d0213f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:11.259998 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d09e4-92cd-4363-a2f7-2eb7d0213f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:11.261427 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d09e4-92cd-4363-a2f7-2eb7d0213f25 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:11.270635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 729/1458] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:13.310239 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc890950-4a2b-4104-b9a1-fef92db5d815 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:13.310791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc890950-4a2b-4104-b9a1-fef92db5d815 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:13.310791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc890950-4a2b-4104-b9a1-fef92db5d815 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:13.312183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 10:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:13.317859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed4d68-7775-4402-a93e-043c027c8b66 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:13.318279 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed4d68-7775-4402-a93e-043c027c8b66 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:13.318443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed4d68-7775-4402-a93e-043c027c8b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:13.336566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-12ed4d68-7775-4402-a93e-043c027c8b66 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:13.337764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-12ed4d68-7775-4402-a93e-043c027c8b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:13.345545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 10:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:13.382990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc890950-4a2b-4104-b9a1-fef92db5d815 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:13.394012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc890950-4a2b-4104-b9a1-fef92db5d815 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:13.397479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4949c6-7066-4641-8837-3908bc3f3902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:13.398101 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4949c6-7066-4641-8837-3908bc3f3902 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:13.398380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4949c6-7066-4641-8837-3908bc3f3902 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:13.398651 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4949c6-7066-4641-8837-3908bc3f3902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:13.407270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4949c6-7066-4641-8837-3908bc3f3902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:13.408718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4949c6-7066-4641-8837-3908bc3f3902 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:13.417852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 731/1461] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:15.478148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c97e-188d-46fe-8e7c-491a4d011b63 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:15.478773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c97e-188d-46fe-8e7c-491a4d011b63 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:15.478773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c97e-188d-46fe-8e7c-491a4d011b63 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:15.480077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 10:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:15.485190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ece250-8028-4b04-aa54-a527c57d71f4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:15.485581 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ece250-8028-4b04-aa54-a527c57d71f4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:15.485744 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ece250-8028-4b04-aa54-a527c57d71f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:15.491674 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=79 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:56:15.504789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-45ece250-8028-4b04-aa54-a527c57d71f4 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:15.505981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-45ece250-8028-4b04-aa54-a527c57d71f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:15.513640 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 10:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:15.551437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c97e-188d-46fe-8e7c-491a4d011b63 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:15.556731 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=73 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:56:15.563113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c97e-188d-46fe-8e7c-491a4d011b63 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:15.567125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16c218-8bbf-4b32-a2e4-f958ca7946ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:15.567777 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16c218-8bbf-4b32-a2e4-f958ca7946ae service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:15.568061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16c218-8bbf-4b32-a2e4-f958ca7946ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:15.568335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16c218-8bbf-4b32-a2e4-f958ca7946ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:15.577037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16c218-8bbf-4b32-a2e4-f958ca7946ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:15.578764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16c218-8bbf-4b32-a2e4-f958ca7946ae service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:15.588541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 732/1464] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:17.629194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c294df-fbc8-4801-9df0-993052138886 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:17.629767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c294df-fbc8-4801-9df0-993052138886 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:17.629767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c294df-fbc8-4801-9df0-993052138886 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:17.631094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 10:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:17.636230 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e962d-3384-4d6a-b9ea-a9882e110cfc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:17.636673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e962d-3384-4d6a-b9ea-a9882e110cfc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:17.636780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e962d-3384-4d6a-b9ea-a9882e110cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:17.654323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-957e962d-3384-4d6a-b9ea-a9882e110cfc None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:17.655517 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-957e962d-3384-4d6a-b9ea-a9882e110cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:17.663362 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 10:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:17.699913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c294df-fbc8-4801-9df0-993052138886 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:17.710512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c294df-fbc8-4801-9df0-993052138886 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:17.714002 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735f613-9c0f-4953-8e8a-3654a45c5e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:17.714635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735f613-9c0f-4953-8e8a-3654a45c5e73 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:17.714917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735f613-9c0f-4953-8e8a-3654a45c5e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:17.715192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735f613-9c0f-4953-8e8a-3654a45c5e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:17.724370 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735f613-9c0f-4953-8e8a-3654a45c5e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:17.725787 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735f613-9c0f-4953-8e8a-3654a45c5e73 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:17.735731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 734/1467] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:19.728459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8b935-7b07-4f48-bb50-3925d9872ec2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:19.729026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8b935-7b07-4f48-bb50-3925d9872ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:19.729026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8b935-7b07-4f48-bb50-3925d9872ec2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:19.730454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 10:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:19.735765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9afd77-2a04-4a0a-b056-2c332a581c30 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:19.736168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9afd77-2a04-4a0a-b056-2c332a581c30 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:19.736319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9afd77-2a04-4a0a-b056-2c332a581c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:19.753954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fe9afd77-2a04-4a0a-b056-2c332a581c30 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:19.755130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fe9afd77-2a04-4a0a-b056-2c332a581c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:19.762808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 10:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:19.799045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8b935-7b07-4f48-bb50-3925d9872ec2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:19.809690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8b935-7b07-4f48-bb50-3925d9872ec2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:19.813234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313af094-b0a4-4b67-80a0-77a7fa48ec44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:19.813871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313af094-b0a4-4b67-80a0-77a7fa48ec44 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:19.814173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313af094-b0a4-4b67-80a0-77a7fa48ec44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:19.814437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313af094-b0a4-4b67-80a0-77a7fa48ec44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:19.824047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313af094-b0a4-4b67-80a0-77a7fa48ec44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:19.825469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313af094-b0a4-4b67-80a0-77a7fa48ec44 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:19.835268 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 735/1470] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:29.748890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=11 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:56:29.831092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=14 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:56:39.523369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22204a86-8821-4461-a923-a528019fdd60 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:39.523997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22204a86-8821-4461-a923-a528019fdd60 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:39.523997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22204a86-8821-4461-a923-a528019fdd60 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:39.525329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 736/1471] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:39.531305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6bcdb-96d3-4914-a76d-b8d12ae2f9ad None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:39.531744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6bcdb-96d3-4914-a76d-b8d12ae2f9ad None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:39.531910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6bcdb-96d3-4914-a76d-b8d12ae2f9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:39.549899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5dc6bcdb-96d3-4914-a76d-b8d12ae2f9ad None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:39.551093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5dc6bcdb-96d3-4914-a76d-b8d12ae2f9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:39.558807 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 736/1472] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:39.605482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d469604-a9f3-4df6-97b1-db9480107a65 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:39.605822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d469604-a9f3-4df6-97b1-db9480107a65 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:39.606107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d469604-a9f3-4df6-97b1-db9480107a65 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:39.607526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 737/1473] 10.222.0.65 () {54 vars in 962 bytes} [Thu Jun 23 10:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:39.612880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5dc17-a465-4b1f-bff6-3909cb71350e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:39.613167 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5dc17-a465-4b1f-bff6-3909cb71350e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:39.613447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5dc17-a465-4b1f-bff6-3909cb71350e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:39.631517 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a3b5dc17-a465-4b1f-bff6-3909cb71350e None None] MFA Rules not processed for user `5af6cc8c56db48beae2550954446c119`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:39.632722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a3b5dc17-a465-4b1f-bff6-3909cb71350e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:39.639259 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 737/1474] 10.222.0.65 () {58 vars in 1059 bytes} [Thu Jun 23 10:56:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:39.646353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d469604-a9f3-4df6-97b1-db9480107a65 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:39.657402 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d469604-a9f3-4df6-97b1-db9480107a65 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:39.660943 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe6d5e-4daf-4e5a-aebc-e9c3ec895181 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af6cc8c56db48beae2550954446c119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:39.661561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe6d5e-4daf-4e5a-aebc-e9c3ec895181 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:39.661845 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe6d5e-4daf-4e5a-aebc-e9c3ec895181 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:39.662124 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe6d5e-4daf-4e5a-aebc-e9c3ec895181 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:39.671429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe6d5e-4daf-4e5a-aebc-e9c3ec895181 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:39.672887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe6d5e-4daf-4e5a-aebc-e9c3ec895181 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:39.682186 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 738/1475] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:41.771950 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842f310-77e7-4889-b75d-f41bb27fde5f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:41.772517 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842f310-77e7-4889-b75d-f41bb27fde5f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:41.772517 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842f310-77e7-4889-b75d-f41bb27fde5f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:41.773868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 738/1476] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:41.779008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a92f4-dec5-4aa6-bdc9-1ac01d5bc0ff None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:41.779950 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a92f4-dec5-4aa6-bdc9-1ac01d5bc0ff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:41.780116 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a92f4-dec5-4aa6-bdc9-1ac01d5bc0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:41.797999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-086a92f4-dec5-4aa6-bdc9-1ac01d5bc0ff None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:41.799189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-086a92f4-dec5-4aa6-bdc9-1ac01d5bc0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:41.806967 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 739/1477] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:41.829804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842f310-77e7-4889-b75d-f41bb27fde5f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:41.842400 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842f310-77e7-4889-b75d-f41bb27fde5f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:41.846727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600725aa-50b4-4dc0-a5a0-1e1eec7316be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:41.847357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600725aa-50b4-4dc0-a5a0-1e1eec7316be admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:41.847637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600725aa-50b4-4dc0-a5a0-1e1eec7316be admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:41.847932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600725aa-50b4-4dc0-a5a0-1e1eec7316be admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:41.848659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600725aa-50b4-4dc0-a5a0-1e1eec7316be admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:41.850433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-600725aa-50b4-4dc0-a5a0-1e1eec7316be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 10:56:41.850433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 23 10:56:41.850433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: Jun 23 10:56:41.851558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600725aa-50b4-4dc0-a5a0-1e1eec7316be admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:41.870610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 739/1478] 10.222.0.65 () {56 vars in 1108 bytes} [Thu Jun 23 10:56:41 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 10:56:43.895295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d3b54-fead-450e-a3c8-85ee033f0e01 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:43.895910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d3b54-fead-450e-a3c8-85ee033f0e01 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:43.895910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d3b54-fead-450e-a3c8-85ee033f0e01 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:43.897260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 740/1479] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:43.902305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889709f6-69c1-4343-81ba-204ab8d37600 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:43.902907 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889709f6-69c1-4343-81ba-204ab8d37600 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:43.902907 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889709f6-69c1-4343-81ba-204ab8d37600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:43.921420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-889709f6-69c1-4343-81ba-204ab8d37600 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:43.922627 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-889709f6-69c1-4343-81ba-204ab8d37600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:43.930473 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 740/1480] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:43.973315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d3b54-fead-450e-a3c8-85ee033f0e01 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:43.987498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d3b54-fead-450e-a3c8-85ee033f0e01 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:43.992996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee71118-a239-4310-be1c-00bc08069de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:43.993630 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71118-a239-4310-be1c-00bc08069de6 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:43.993909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71118-a239-4310-be1c-00bc08069de6 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:43.994184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71118-a239-4310-be1c-00bc08069de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:44.003029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee71118-a239-4310-be1c-00bc08069de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:44.004619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee71118-a239-4310-be1c-00bc08069de6 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:44.014393 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 741/1481] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:46.200281 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68586ad-bdcd-4277-a217-42a105f54f7e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:46.201326 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68586ad-bdcd-4277-a217-42a105f54f7e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:46.201442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68586ad-bdcd-4277-a217-42a105f54f7e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:46.202777 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 741/1482] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:56:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:46.208041 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca49c7-2c84-4f40-85e5-2f99c2bf93ac None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:46.208491 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca49c7-2c84-4f40-85e5-2f99c2bf93ac None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:46.208600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca49c7-2c84-4f40-85e5-2f99c2bf93ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:46.226386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-abca49c7-2c84-4f40-85e5-2f99c2bf93ac None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:46.227589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-abca49c7-2c84-4f40-85e5-2f99c2bf93ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:46.235213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 742/1483] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:46.272110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68586ad-bdcd-4277-a217-42a105f54f7e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:46.282646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68586ad-bdcd-4277-a217-42a105f54f7e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:46.286125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6dc60b-d85f-417b-bc0a-4a1e38c2ce74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:46.286749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6dc60b-d85f-417b-bc0a-4a1e38c2ce74 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:46.287028 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6dc60b-d85f-417b-bc0a-4a1e38c2ce74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:46.287300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6dc60b-d85f-417b-bc0a-4a1e38c2ce74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:46.296339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6dc60b-d85f-417b-bc0a-4a1e38c2ce74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:46.297772 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6dc60b-d85f-417b-bc0a-4a1e38c2ce74 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:46.307163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 742/1484] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:48.367837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76d985-7671-4e7a-af59-b4253d0c617e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:48.368287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76d985-7671-4e7a-af59-b4253d0c617e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:48.368391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76d985-7671-4e7a-af59-b4253d0c617e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:48.369759 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 743/1485] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:48.375733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3b85c-fceb-4be3-b9cd-3c9d8a5c02cb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:48.376143 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3b85c-fceb-4be3-b9cd-3c9d8a5c02cb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:48.376292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3b85c-fceb-4be3-b9cd-3c9d8a5c02cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:48.393513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a8b3b85c-fceb-4be3-b9cd-3c9d8a5c02cb None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:48.394696 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a8b3b85c-fceb-4be3-b9cd-3c9d8a5c02cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:48.402372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 743/1486] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:48.440268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76d985-7671-4e7a-af59-b4253d0c617e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:48.450715 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76d985-7671-4e7a-af59-b4253d0c617e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:48.454202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c292a81-cf6a-4817-8e17-765ac3a3e0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:48.454823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c292a81-cf6a-4817-8e17-765ac3a3e0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:48.455104 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c292a81-cf6a-4817-8e17-765ac3a3e0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:48.455377 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c292a81-cf6a-4817-8e17-765ac3a3e0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:48.464034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c292a81-cf6a-4817-8e17-765ac3a3e0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:48.465441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c292a81-cf6a-4817-8e17-765ac3a3e0a0 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:48.475243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 744/1487] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:50.575961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3347a58-a8b9-42fb-be5a-e6f030d4432c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:50.576513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3347a58-a8b9-42fb-be5a-e6f030d4432c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:50.576513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3347a58-a8b9-42fb-be5a-e6f030d4432c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:50.577895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 744/1488] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:50.583025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea918b-0ae0-4060-9bae-5aeab7bd65da None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:50.583410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea918b-0ae0-4060-9bae-5aeab7bd65da None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:50.583587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea918b-0ae0-4060-9bae-5aeab7bd65da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:50.601359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f5ea918b-0ae0-4060-9bae-5aeab7bd65da None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:50.602475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5ea918b-0ae0-4060-9bae-5aeab7bd65da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:50.610754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 745/1489] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:50.650354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3347a58-a8b9-42fb-be5a-e6f030d4432c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:50.661260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3347a58-a8b9-42fb-be5a-e6f030d4432c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:50.664799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c2677-a917-44c5-add5-2e8fd9cc0d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:50.665441 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c2677-a917-44c5-add5-2e8fd9cc0d15 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:50.665727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c2677-a917-44c5-add5-2e8fd9cc0d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:50.666006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c2677-a917-44c5-add5-2e8fd9cc0d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:50.675303 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c2677-a917-44c5-add5-2e8fd9cc0d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:50.676778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c2677-a917-44c5-add5-2e8fd9cc0d15 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:50.687042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 745/1490] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:56:52.897195 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837397de-aa48-4661-a01b-598f202196cb None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:52.897769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837397de-aa48-4661-a01b-598f202196cb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:52.897769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837397de-aa48-4661-a01b-598f202196cb None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:52.899108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 746/1491] 10.222.0.65 () {54 vars in 926 bytes} [Thu Jun 23 10:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:56:52.905003 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98202101-fd4f-4c89-a5bb-ff931500ace7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:52.905455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98202101-fd4f-4c89-a5bb-ff931500ace7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:52.905569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98202101-fd4f-4c89-a5bb-ff931500ace7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:52.923044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-98202101-fd4f-4c89-a5bb-ff931500ace7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:56:52.924257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-98202101-fd4f-4c89-a5bb-ff931500ace7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:56:52.931903 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 746/1492] 10.222.0.65 () {58 vars in 1023 bytes} [Thu Jun 23 10:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:56:52.970730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837397de-aa48-4661-a01b-598f202196cb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:56:52.981193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837397de-aa48-4661-a01b-598f202196cb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:56:52.984646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8515d00-ff1a-48d3-baf1-c0b43ead8997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:56:52.985278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8515d00-ff1a-48d3-baf1-c0b43ead8997 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:56:52.985556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8515d00-ff1a-48d3-baf1-c0b43ead8997 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:56:52.985828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8515d00-ff1a-48d3-baf1-c0b43ead8997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:56:52.994566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8515d00-ff1a-48d3-baf1-c0b43ead8997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:56:52.996028 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8515d00-ff1a-48d3-baf1-c0b43ead8997 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:56:53.006052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 747/1493] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:57:02.918090 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=33 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:57:03.001910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=33 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:58:17.138969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04357f29-b973-451c-a99e-2c1b1770135d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.139516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04357f29-b973-451c-a99e-2c1b1770135d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.139516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04357f29-b973-451c-a99e-2c1b1770135d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.157054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-04357f29-b973-451c-a99e-2c1b1770135d None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:58:17.158241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-04357f29-b973-451c-a99e-2c1b1770135d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:58:17.166482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 747/1494] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 10:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:58:17.175507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded202c-2e47-45b1-a815-6383b82cc965 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.176073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded202c-2e47-45b1-a815-6383b82cc965 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.176073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded202c-2e47-45b1-a815-6383b82cc965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.194096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0ded202c-2e47-45b1-a815-6383b82cc965 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:58:17.195319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0ded202c-2e47-45b1-a815-6383b82cc965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:58:17.203034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 748/1495] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 10:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:58:17.207590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04357f29-b973-451c-a99e-2c1b1770135d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:58:17.217323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04357f29-b973-451c-a99e-2c1b1770135d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:58:17.221949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fff23d-d685-4d7e-92b2-93ebcae4cf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:58:17.222607 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fff23d-d685-4d7e-92b2-93ebcae4cf9d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.222889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fff23d-d685-4d7e-92b2-93ebcae4cf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.223167 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fff23d-d685-4d7e-92b2-93ebcae4cf9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.223918 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fff23d-d685-4d7e-92b2-93ebcae4cf9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:58:17.224258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fff23d-d685-4d7e-92b2-93ebcae4cf9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:58:17.225294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-a2fff23d-d685-4d7e-92b2-93ebcae4cf9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 10:58:17.225294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 23 10:58:17.225294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: Jun 23 10:58:17.226721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fff23d-d685-4d7e-92b2-93ebcae4cf9d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:58:17.238211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 748/1496] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 10:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:58:17.246870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded202c-2e47-45b1-a815-6383b82cc965 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:58:17.255878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded202c-2e47-45b1-a815-6383b82cc965 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:58:17.260354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaad0ca-3eb8-4a2d-82b9-82fb989a1582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:58:17.260994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaad0ca-3eb8-4a2d-82b9-82fb989a1582 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.261276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaad0ca-3eb8-4a2d-82b9-82fb989a1582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.261551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaad0ca-3eb8-4a2d-82b9-82fb989a1582 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.262249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaad0ca-3eb8-4a2d-82b9-82fb989a1582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:58:17.264049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-cdaad0ca-3eb8-4a2d-82b9-82fb989a1582 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 10:58:17.264049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 23 10:58:17.264049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: Jun 23 10:58:17.265233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaad0ca-3eb8-4a2d-82b9-82fb989a1582 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:58:17.300820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 749/1497] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 10:58:17 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:58:17.305673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fff23d-d685-4d7e-92b2-93ebcae4cf9d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:58:17.315471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fff23d-d685-4d7e-92b2-93ebcae4cf9d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:58:17.370881 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a3513-b103-477c-ba71-7cec47b3cca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:58:17.371486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3513-b103-477c-ba71-7cec47b3cca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.371805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3513-b103-477c-ba71-7cec47b3cca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.372089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3513-b103-477c-ba71-7cec47b3cca3 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.372786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a3513-b103-477c-ba71-7cec47b3cca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:58:17.373795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-049a3513-b103-477c-ba71-7cec47b3cca3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 10:58:17.373795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 23 10:58:17.373795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: Jun 23 10:58:17.374995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a3513-b103-477c-ba71-7cec47b3cca3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:58:17.395220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 749/1498] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 10:58:17 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:58:17.400075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaad0ca-3eb8-4a2d-82b9-82fb989a1582 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:58:17.410121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaad0ca-3eb8-4a2d-82b9-82fb989a1582 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:58:17.415233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902086-88f3-4417-9d57-4e1c4ef84c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:58:17.415917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902086-88f3-4417-9d57-4e1c4ef84c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.416203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902086-88f3-4417-9d57-4e1c4ef84c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.416477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902086-88f3-4417-9d57-4e1c4ef84c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.417192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74902086-88f3-4417-9d57-4e1c4ef84c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:58:17.418192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-74902086-88f3-4417-9d57-4e1c4ef84c91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 10:58:17.418192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 23 10:58:17.418192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: Jun 23 10:58:17.419086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74902086-88f3-4417-9d57-4e1c4ef84c91 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:58:17.431519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 750/1499] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 10:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 10:58:17.435628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a3513-b103-477c-ba71-7cec47b3cca3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:58:17.445385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a3513-b103-477c-ba71-7cec47b3cca3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:58:17.449947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82634fea-991d-439e-8d21-7eec917df7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:58:17.450597 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82634fea-991d-439e-8d21-7eec917df7cf admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.450879 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82634fea-991d-439e-8d21-7eec917df7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.451160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82634fea-991d-439e-8d21-7eec917df7cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.451922 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82634fea-991d-439e-8d21-7eec917df7cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:58:17.452929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-82634fea-991d-439e-8d21-7eec917df7cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 10:58:17.452929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 23 10:58:17.452929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: Jun 23 10:58:17.453817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82634fea-991d-439e-8d21-7eec917df7cf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:58:17.467233 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 750/1500] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 10:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 10:58:17.471128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902086-88f3-4417-9d57-4e1c4ef84c91 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:58:17.481527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902086-88f3-4417-9d57-4e1c4ef84c91 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:58:17.485929 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332107de-32dd-42f8-a7e4-2aaf13f1cd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:58:17.486784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332107de-32dd-42f8-a7e4-2aaf13f1cd61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.487060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332107de-32dd-42f8-a7e4-2aaf13f1cd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.487328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332107de-32dd-42f8-a7e4-2aaf13f1cd61 admin admin] PATH_INFO: `/v3/projects/4bd71745a2614eb689f092ab5680a70c/users/4cd5d6415a92435c81130e375d8e0fbf/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.502182 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332107de-32dd-42f8-a7e4-2aaf13f1cd61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd71745a2614eb689f092ab5680a70c, user_id=4cd5d6415a92435c81130e375d8e0fbf, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:58:17.503193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-332107de-32dd-42f8-a7e4-2aaf13f1cd61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 10:58:17.503193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 23 10:58:17.503193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: Jun 23 10:58:17.505133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332107de-32dd-42f8-a7e4-2aaf13f1cd61 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:58:17.517547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 751/1501] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 10:58:17 2022] PUT /identity/v3/projects/4bd71745a2614eb689f092ab5680a70c/users/4cd5d6415a92435c81130e375d8e0fbf/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:58:17.528990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b9059-6218-4994-bf28-2a3c8f0da0a7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.529285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b9059-6218-4994-bf28-2a3c8f0da0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.529562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b9059-6218-4994-bf28-2a3c8f0da0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.556373 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ac0b9059-6218-4994-bf28-2a3c8f0da0a7 None None] MFA Rules not processed for user `4cd5d6415a92435c81130e375d8e0fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:58:17.557559 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ac0b9059-6218-4994-bf28-2a3c8f0da0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:58:17.619890 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 751/1502] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 10:58:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:58:17.632686 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eef1b-fa17-49fd-8609-71042bbbd652 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.632976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eef1b-fa17-49fd-8609-71042bbbd652 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.633251 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eef1b-fa17-49fd-8609-71042bbbd652 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.634644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 752/1503] 10.222.0.65 () {54 vars in 963 bytes} [Thu Jun 23 10:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:58:17.639294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e3036-a6b9-4c89-ae07-ea502d3f6483 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.639591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e3036-a6b9-4c89-ae07-ea502d3f6483 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.639912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e3036-a6b9-4c89-ae07-ea502d3f6483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.657975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5b9e3036-a6b9-4c89-ae07-ea502d3f6483 None None] MFA Rules not processed for user `3663612647934677b83fb42aaf06d6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:58:17.659223 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5b9e3036-a6b9-4c89-ae07-ea502d3f6483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:58:17.701525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:58:17.709643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 752/1504] 10.222.0.65 () {58 vars in 1060 bytes} [Thu Jun 23 10:58:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:58:17.717691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870eef1b-fa17-49fd-8609-71042bbbd652 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:58:17.728596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870eef1b-fa17-49fd-8609-71042bbbd652 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:58:17.732032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18940d59-2add-424d-8a83-acc1f4e03b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:58:17.732654 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18940d59-2add-424d-8a83-acc1f4e03b33 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.732934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18940d59-2add-424d-8a83-acc1f4e03b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.733206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18940d59-2add-424d-8a83-acc1f4e03b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.741192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18940d59-2add-424d-8a83-acc1f4e03b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:58:17.742596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18940d59-2add-424d-8a83-acc1f4e03b33 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:58:17.751353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 753/1505] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:58:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:58:17.793066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b387-f77a-4f19-854e-f381c6c6898d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.793380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b387-f77a-4f19-854e-f381c6c6898d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.793663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b387-f77a-4f19-854e-f381c6c6898d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.795079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 753/1506] 10.222.0.65 () {52 vars in 829 bytes} [Thu Jun 23 10:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:58:17.798570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18940d59-2add-424d-8a83-acc1f4e03b33 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:58:17.858591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18940d59-2add-424d-8a83-acc1f4e03b33 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:58:17.863092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:58:17.863986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.864272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.864548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] PATH_INFO: `/v3/users/4cd5d6415a92435c81130e375d8e0fbf` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:17.865718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cd5d6415a92435c81130e375d8e0fbf)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:58:17.866719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 10:58:17.866719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 23 10:58:17.866719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: Jun 23 10:58:17.867957 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:58:17.930804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cd5d6415a92435c81130e375d8e0fbf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 10:58:17.931586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cd5d6415a92435c81130e375d8e0fbf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 10:58:17.940664 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cd5d6415a92435c81130e375d8e0fbf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 10:58:17.958958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cd5d6415a92435c81130e375d8e0fbf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 10:58:17.968005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:58:17.973105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cd5d6415a92435c81130e375d8e0fbf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 10:58:17.980140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 754/1507] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 10:58:17 2022] DELETE /identity/v3/users/4cd5d6415a92435c81130e375d8e0fbf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:58:17.984674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72b85c-25dc-4293-ad75-4604fb6e83f8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:17.984993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72b85c-25dc-4293-ad75-4604fb6e83f8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:17.985276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72b85c-25dc-4293-ad75-4604fb6e83f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:18.002755 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0d72b85c-25dc-4293-ad75-4604fb6e83f8 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:58:18.003987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0d72b85c-25dc-4293-ad75-4604fb6e83f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:58:18.061874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 754/1508] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 10:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:58:18.072294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:58:18.073674 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:58:18.131313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f17cf-8e56-4e03-8ff8-aabd7912020d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:58:18.135422 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f38fe-c9f9-4159-be12-4ad6abf02003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:58:18.136101 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f38fe-c9f9-4159-be12-4ad6abf02003 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:18.136389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f38fe-c9f9-4159-be12-4ad6abf02003 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:18.136671 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f38fe-c9f9-4159-be12-4ad6abf02003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:18.145597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f38fe-c9f9-4159-be12-4ad6abf02003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:58:18.147035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f38fe-c9f9-4159-be12-4ad6abf02003 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:58:18.156391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 755/1509] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:58:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:58:18.491764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72b85c-25dc-4293-ad75-4604fb6e83f8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:58:18.494220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0d72b85c-25dc-4293-ad75-4604fb6e83f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:58:18.506339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72b85c-25dc-4293-ad75-4604fb6e83f8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:58:18.511061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:58:18.511952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:58:18.518071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:58:18.518375 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] PATH_INFO: `/v3/projects/4bd71745a2614eb689f092ab5680a70c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:58:18.519634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd71745a2614eb689f092ab5680a70c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:58:18.520726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 10:58:18.520726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 23 10:58:18.520726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: Jun 23 10:58:18.521909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:58:18.581830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd71745a2614eb689f092ab5680a70c', 'request_id': 'req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 10:58:18.594706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd71745a2614eb689f092ab5680a70c', 'request_id': 'req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 10:58:18.600980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd71745a2614eb689f092ab5680a70c', 'request_id': 'req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 10:58:18.616659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] The token cache is being invalidate because project 4bd71745a2614eb689f092ab5680a70c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 10:58:18.617586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 10:58:18.618219 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4f0c7d25-0eb6-41f7-98c3-f6af88047a03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 10:58:18.620174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 755/1510] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 10:58:18 2022] DELETE /identity/v3/projects/4bd71745a2614eb689f092ab5680a70c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:58:28.152218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=29 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:58:28.613501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:59:57.727726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88404b-ec20-4844-a422-00eac477324f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:59:57.728335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88404b-ec20-4844-a422-00eac477324f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:59:57.728335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88404b-ec20-4844-a422-00eac477324f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:59:57.729694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 10:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:59:57.734403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3ef9e-c842-4af6-becb-603ac51b823e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:59:57.734838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3ef9e-c842-4af6-becb-603ac51b823e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:59:57.734959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3ef9e-c842-4af6-becb-603ac51b823e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:59:57.753396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-78f3ef9e-c842-4af6-becb-603ac51b823e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:59:57.754583 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-78f3ef9e-c842-4af6-becb-603ac51b823e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:59:57.811552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 10:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:59:57.967313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88404b-ec20-4844-a422-00eac477324f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:59:57.969060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0a88404b-ec20-4844-a422-00eac477324f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:59:58.043407 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88404b-ec20-4844-a422-00eac477324f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:59:58.048741 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41fe26-8928-44bf-9fc0-dcf8fa372b55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:59:58.049381 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41fe26-8928-44bf-9fc0-dcf8fa372b55 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:59:58.049661 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41fe26-8928-44bf-9fc0-dcf8fa372b55 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:59:58.049935 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41fe26-8928-44bf-9fc0-dcf8fa372b55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:59:58.057842 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41fe26-8928-44bf-9fc0-dcf8fa372b55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:59:58.059425 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41fe26-8928-44bf-9fc0-dcf8fa372b55 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:59:58.068542 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 757/1513] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:59:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:59:58.897387 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf46a2f-f409-4783-92e8-7abd4f813087 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:59:58.897969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf46a2f-f409-4783-92e8-7abd4f813087 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:59:58.897969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf46a2f-f409-4783-92e8-7abd4f813087 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:59:58.899394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 10:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:59:58.904152 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32299be-ef5b-4272-9416-b42a4beb3d0f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:59:58.904588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32299be-ef5b-4272-9416-b42a4beb3d0f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:59:58.904734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32299be-ef5b-4272-9416-b42a4beb3d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:59:58.925191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c32299be-ef5b-4272-9416-b42a4beb3d0f None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:59:58.926408 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c32299be-ef5b-4272-9416-b42a4beb3d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:59:58.934930 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 10:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 10:59:59.087264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf46a2f-f409-4783-92e8-7abd4f813087 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:59:59.102285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf46a2f-f409-4783-92e8-7abd4f813087 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:59:59.109385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f01b2-5dfa-4bcc-8b13-0dc2471e5680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:59:59.110040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f01b2-5dfa-4bcc-8b13-0dc2471e5680 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:59:59.110328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f01b2-5dfa-4bcc-8b13-0dc2471e5680 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:59:59.110605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f01b2-5dfa-4bcc-8b13-0dc2471e5680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:59:59.119972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f01b2-5dfa-4bcc-8b13-0dc2471e5680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:59:59.121585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f01b2-5dfa-4bcc-8b13-0dc2471e5680 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:59:59.131137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 758/1516] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 10:59:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 10:59:59.878420 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64df34b-f622-4f49-94b0-11119e63e52e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:59:59.878710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64df34b-f622-4f49-94b0-11119e63e52e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:59:59.878992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64df34b-f622-4f49-94b0-11119e63e52e None None] PATH_INFO: `/v3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:59:59.880345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 759/1517] 10.222.0.60 () {54 vars in 925 bytes} [Thu Jun 23 10:59:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:59:59.915255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5f1e8-1495-47f0-93fe-f987f2694470 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:59:59.915776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5f1e8-1495-47f0-93fe-f987f2694470 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:59:59.915896 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5f1e8-1495-47f0-93fe-f987f2694470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:59:59.933877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-75d5f1e8-1495-47f0-93fe-f987f2694470 None None] MFA Rules not processed for user `18fc0e7956b14a49beab025922e8ff81`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:59:59.935098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-75d5f1e8-1495-47f0-93fe-f987f2694470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:59:59.998680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 759/1518] 10.222.0.60 () {58 vars in 1015 bytes} [Thu Jun 23 10:59:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:00:08.918076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=37 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:00:09.357292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d2b67-6189-4e7e-981a-5ddf97e325fa None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:09.357572 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d2b67-6189-4e7e-981a-5ddf97e325fa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:09.357847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d2b67-6189-4e7e-981a-5ddf97e325fa None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:09.359215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 760/1519] 10.222.0.65 () {54 vars in 960 bytes} [Thu Jun 23 11:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:00:09.364046 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f34c2a-51a9-4cbb-8f8c-9e874588e26c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:09.364522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f34c2a-51a9-4cbb-8f8c-9e874588e26c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:09.364648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f34c2a-51a9-4cbb-8f8c-9e874588e26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:09.383864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e6f34c2a-51a9-4cbb-8f8c-9e874588e26c None None] MFA Rules not processed for user `18fc0e7956b14a49beab025922e8ff81`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:00:09.385069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e6f34c2a-51a9-4cbb-8f8c-9e874588e26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:00:09.393495 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 760/1520] 10.222.0.65 () {58 vars in 1057 bytes} [Thu Jun 23 11:00:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:00:09.400932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d2b67-6189-4e7e-981a-5ddf97e325fa None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:00:09.414670 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d2b67-6189-4e7e-981a-5ddf97e325fa None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:00:09.419525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383964b-2367-4c22-baec-d4ec51fcfa32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:00:09.420209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383964b-2367-4c22-baec-d4ec51fcfa32 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:09.420490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383964b-2367-4c22-baec-d4ec51fcfa32 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:09.420763 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383964b-2367-4c22-baec-d4ec51fcfa32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:09.429479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383964b-2367-4c22-baec-d4ec51fcfa32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:00:09.431067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383964b-2367-4c22-baec-d4ec51fcfa32 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:00:09.440869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 761/1521] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:00:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:00:10.369422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361fe6f-f00d-4ff8-a7d4-f41159dc07ce None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:10.369975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361fe6f-f00d-4ff8-a7d4-f41159dc07ce None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:10.370092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361fe6f-f00d-4ff8-a7d4-f41159dc07ce None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:10.371476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:00:10.376313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dc572-0e3c-4656-ac52-82ef6fb95205 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:10.376736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dc572-0e3c-4656-ac52-82ef6fb95205 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:10.376897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dc572-0e3c-4656-ac52-82ef6fb95205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:10.395098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-031dc572-0e3c-4656-ac52-82ef6fb95205 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:00:10.396354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-031dc572-0e3c-4656-ac52-82ef6fb95205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:00:10.404787 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:00:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:00:10.558510 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361fe6f-f00d-4ff8-a7d4-f41159dc07ce None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:00:10.573589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361fe6f-f00d-4ff8-a7d4-f41159dc07ce None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:00:10.579091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1c2f4-0fd0-4bea-8640-c4ce944afcc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:00:10.579776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1c2f4-0fd0-4bea-8640-c4ce944afcc1 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:10.580060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1c2f4-0fd0-4bea-8640-c4ce944afcc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:10.580339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1c2f4-0fd0-4bea-8640-c4ce944afcc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:10.588458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1c2f4-0fd0-4bea-8640-c4ce944afcc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:00:10.590055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1c2f4-0fd0-4bea-8640-c4ce944afcc1 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:00:10.599368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 762/1524] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:00:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:00:11.414323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993bbd7-8750-46e6-b211-6b5da34beaa5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:11.414863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993bbd7-8750-46e6-b211-6b5da34beaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:11.414863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993bbd7-8750-46e6-b211-6b5da34beaa5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:11.416232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:00:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:00:11.421032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c777e0a-57f2-448a-a9a6-fc7531c5b62f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:11.421410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c777e0a-57f2-448a-a9a6-fc7531c5b62f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:11.421596 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c777e0a-57f2-448a-a9a6-fc7531c5b62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:11.439031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0c777e0a-57f2-448a-a9a6-fc7531c5b62f None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:00:11.440258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0c777e0a-57f2-448a-a9a6-fc7531c5b62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:00:11.448040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:00:11.597740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993bbd7-8750-46e6-b211-6b5da34beaa5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:00:11.612424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993bbd7-8750-46e6-b211-6b5da34beaa5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:00:11.617697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0077eeb-5e9f-4f29-9dc7-dadbb29461ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:00:11.618319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0077eeb-5e9f-4f29-9dc7-dadbb29461ee service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:11.618851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0077eeb-5e9f-4f29-9dc7-dadbb29461ee service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:11.619124 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0077eeb-5e9f-4f29-9dc7-dadbb29461ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:11.626988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0077eeb-5e9f-4f29-9dc7-dadbb29461ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:00:11.628582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0077eeb-5e9f-4f29-9dc7-dadbb29461ee service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:00:11.638186 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 764/1527] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:00:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:00:15.507261 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45c814-f7d3-4d50-ac38-f8ba6461ac22 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:15.507825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45c814-f7d3-4d50-ac38-f8ba6461ac22 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:15.507825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45c814-f7d3-4d50-ac38-f8ba6461ac22 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:15.509176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:00:15.514652 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a7490-3842-41fb-955b-58bd1cbe7311 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:15.515078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a7490-3842-41fb-955b-58bd1cbe7311 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:15.515208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a7490-3842-41fb-955b-58bd1cbe7311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:15.532972 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cd3a7490-3842-41fb-955b-58bd1cbe7311 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:00:15.534155 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cd3a7490-3842-41fb-955b-58bd1cbe7311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:00:15.542424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:00:15.579334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45c814-f7d3-4d50-ac38-f8ba6461ac22 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:00:15.581026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dc45c814-f7d3-4d50-ac38-f8ba6461ac22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:00:15.637152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45c814-f7d3-4d50-ac38-f8ba6461ac22 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:00:15.640618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385512c-e888-4564-ab72-48f91dca346b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:00:15.641253 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385512c-e888-4564-ab72-48f91dca346b service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:15.641532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385512c-e888-4564-ab72-48f91dca346b service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:15.641807 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385512c-e888-4564-ab72-48f91dca346b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:15.650524 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385512c-e888-4564-ab72-48f91dca346b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:00:15.651966 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385512c-e888-4564-ab72-48f91dca346b service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:00:15.656799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:00:15.662750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 765/1530] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:00:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:00:17.679498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb233f-58b9-4b6d-9720-44fbe54b994f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:17.680065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb233f-58b9-4b6d-9720-44fbe54b994f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:17.680065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb233f-58b9-4b6d-9720-44fbe54b994f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:17.681440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:00:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:00:17.686483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc06956-db8e-4f45-b53a-833f3b1874f4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:17.686882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc06956-db8e-4f45-b53a-833f3b1874f4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:17.687039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc06956-db8e-4f45-b53a-833f3b1874f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:17.704722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7dc06956-db8e-4f45-b53a-833f3b1874f4 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:00:17.705902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7dc06956-db8e-4f45-b53a-833f3b1874f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:00:17.713550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:00:17.749827 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb233f-58b9-4b6d-9720-44fbe54b994f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:00:17.760420 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb233f-58b9-4b6d-9720-44fbe54b994f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:00:17.763831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6d0e9-c46f-4e96-a3ec-383a1ba34f2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:00:17.764453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6d0e9-c46f-4e96-a3ec-383a1ba34f2b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:17.764738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6d0e9-c46f-4e96-a3ec-383a1ba34f2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:17.765011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6d0e9-c46f-4e96-a3ec-383a1ba34f2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:17.773561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6d0e9-c46f-4e96-a3ec-383a1ba34f2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:00:17.774961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6d0e9-c46f-4e96-a3ec-383a1ba34f2b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:00:17.784049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 767/1533] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:00:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:00:21.330065 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a0eaa-0849-49db-8678-27ef1aaa081e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:21.330629 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a0eaa-0849-49db-8678-27ef1aaa081e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:21.330629 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a0eaa-0849-49db-8678-27ef1aaa081e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:21.332060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:00:21.337091 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8ebef-3f23-4bc3-8c20-847160aae13e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:21.337533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8ebef-3f23-4bc3-8c20-847160aae13e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:21.337641 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8ebef-3f23-4bc3-8c20-847160aae13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:21.355173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2cf8ebef-3f23-4bc3-8c20-847160aae13e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:00:21.356313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2cf8ebef-3f23-4bc3-8c20-847160aae13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:00:21.363933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:00:21.660745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a0eaa-0849-49db-8678-27ef1aaa081e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:00:21.662941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-098a0eaa-0849-49db-8678-27ef1aaa081e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:00:21.718666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a0eaa-0849-49db-8678-27ef1aaa081e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:00:21.722185 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacb7b3-0aef-4b20-8b65-558122b2221b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:00:21.722849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacb7b3-0aef-4b20-8b65-558122b2221b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:21.723133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacb7b3-0aef-4b20-8b65-558122b2221b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:21.723414 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacb7b3-0aef-4b20-8b65-558122b2221b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:21.732784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacb7b3-0aef-4b20-8b65-558122b2221b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:00:21.734211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacb7b3-0aef-4b20-8b65-558122b2221b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:00:21.744305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 768/1536] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:00:23.771184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29ff24-9c10-4583-80ed-87f9bba3dbf1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:23.771776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29ff24-9c10-4583-80ed-87f9bba3dbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:23.771878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29ff24-9c10-4583-80ed-87f9bba3dbf1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:23.773253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:00:23.778406 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76761e-75ce-42b1-8314-36f5cec637c6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:23.778803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76761e-75ce-42b1-8314-36f5cec637c6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:23.778968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76761e-75ce-42b1-8314-36f5cec637c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:23.797359 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3f76761e-75ce-42b1-8314-36f5cec637c6 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:00:23.798594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3f76761e-75ce-42b1-8314-36f5cec637c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:00:23.807019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:00:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:00:24.105221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29ff24-9c10-4583-80ed-87f9bba3dbf1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:00:24.106931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ad29ff24-9c10-4583-80ed-87f9bba3dbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:00:24.119795 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29ff24-9c10-4583-80ed-87f9bba3dbf1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:00:24.123253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784f50c-ab1c-4c88-971c-a2a77292d378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:00:24.123911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784f50c-ab1c-4c88-971c-a2a77292d378 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:00:24.124193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784f50c-ab1c-4c88-971c-a2a77292d378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:00:24.124464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784f50c-ab1c-4c88-971c-a2a77292d378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:00:24.133140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784f50c-ab1c-4c88-971c-a2a77292d378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:00:24.134536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784f50c-ab1c-4c88-971c-a2a77292d378 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:00:24.143667 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 770/1539] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:00:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:00:33.792015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=32 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:00:34.139596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=33 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:51.481278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3bf63-20ad-4785-bcd8-44273f2b03bd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.482545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3bf63-20ad-4785-bcd8-44273f2b03bd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.483068 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3bf63-20ad-4785-bcd8-44273f2b03bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.505096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9bb3bf63-20ad-4785-bcd8-44273f2b03bd None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:51.506330 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9bb3bf63-20ad-4785-bcd8-44273f2b03bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:51.518841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 770/1540] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:51.529835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb975e5-8ac5-4e34-b020-263ad0ccc295 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.530416 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb975e5-8ac5-4e34-b020-263ad0ccc295 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.530650 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb975e5-8ac5-4e34-b020-263ad0ccc295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.556667 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-afb975e5-8ac5-4e34-b020-263ad0ccc295 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:51.558016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-afb975e5-8ac5-4e34-b020-263ad0ccc295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:51.569410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 771/1541] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:51.575052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3bf63-20ad-4785-bcd8-44273f2b03bd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:51.588962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3bf63-20ad-4785-bcd8-44273f2b03bd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:51.597399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe5d6b-a69a-4b0d-a99e-8565f59e4ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:51.598092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe5d6b-a69a-4b0d-a99e-8565f59e4ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.598380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe5d6b-a69a-4b0d-a99e-8565f59e4ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.598661 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe5d6b-a69a-4b0d-a99e-8565f59e4ed6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.599416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe5d6b-a69a-4b0d-a99e-8565f59e4ed6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:01:51.599763 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe5d6b-a69a-4b0d-a99e-8565f59e4ed6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:51.601463 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe5d6b-a69a-4b0d-a99e-8565f59e4ed6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:51.626385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 771/1542] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:01:51.634883 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb975e5-8ac5-4e34-b020-263ad0ccc295 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:51.646341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77523209-ced0-400e-b824-e89ce4989b59 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.646686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77523209-ced0-400e-b824-e89ce4989b59 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.646894 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77523209-ced0-400e-b824-e89ce4989b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.650603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb975e5-8ac5-4e34-b020-263ad0ccc295 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:51.662166 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa253a-151c-43d2-afc1-2f42c6126358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:51.662850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa253a-151c-43d2-afc1-2f42c6126358 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.663136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa253a-151c-43d2-afc1-2f42c6126358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.663448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa253a-151c-43d2-afc1-2f42c6126358 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.665882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa253a-151c-43d2-afc1-2f42c6126358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:51.667852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa253a-151c-43d2-afc1-2f42c6126358 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:51.676269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-77523209-ced0-400e-b824-e89ce4989b59 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:51.677484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-77523209-ced0-400e-b824-e89ce4989b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:51.698639 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 772/1543] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:51.700149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0c8d7-d370-4955-8612-367c0d2ae9d6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.700437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0c8d7-d370-4955-8612-367c0d2ae9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.700711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0c8d7-d370-4955-8612-367c0d2ae9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.720335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 772/1544] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:01:51 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:51.721684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc62c6-1d3d-4f00-a721-355ebf4c1ef4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.721983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc62c6-1d3d-4f00-a721-355ebf4c1ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.722264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc62c6-1d3d-4f00-a721-355ebf4c1ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.731081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-70a0c8d7-d370-4955-8612-367c0d2ae9d6 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:51.732348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-70a0c8d7-d370-4955-8612-367c0d2ae9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:51.761292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ebfc62c6-1d3d-4f00-a721-355ebf4c1ef4 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:51.762577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ebfc62c6-1d3d-4f00-a721-355ebf4c1ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:51.847276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 773/1545] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:51.847959 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc62c6-1d3d-4f00-a721-355ebf4c1ef4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:51.866335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc62c6-1d3d-4f00-a721-355ebf4c1ef4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:51.878097 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 773/1546] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:51.880271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37e0e1-0976-4e93-a7b2-32d9cef09349 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.880559 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37e0e1-0976-4e93-a7b2-32d9cef09349 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.880839 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37e0e1-0976-4e93-a7b2-32d9cef09349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.885098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7811a4d-1ce8-4c74-840f-ff1cd9269d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:51.885765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7811a4d-1ce8-4c74-840f-ff1cd9269d52 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.886043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7811a4d-1ce8-4c74-840f-ff1cd9269d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.886315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7811a4d-1ce8-4c74-840f-ff1cd9269d52 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.887013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7811a4d-1ce8-4c74-840f-ff1cd9269d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:51.889657 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-f7811a4d-1ce8-4c74-840f-ff1cd9269d52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:01:51.889657 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 23 11:01:51.889657 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: Jun 23 11:01:51.890891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7811a4d-1ce8-4c74-840f-ff1cd9269d52 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:51.913089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5d37e0e1-0976-4e93-a7b2-32d9cef09349 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:51.914669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5d37e0e1-0976-4e93-a7b2-32d9cef09349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:51.918136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 774/1547] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:01:51 2022] POST /identity/v3/users => generated 288 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:51.919635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873870f-5f4f-47bc-8401-e80bd5e8221f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.920219 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873870f-5f4f-47bc-8401-e80bd5e8221f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.920734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873870f-5f4f-47bc-8401-e80bd5e8221f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.927885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 774/1548] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:51.928856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37e0e1-0976-4e93-a7b2-32d9cef09349 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:51.941904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37e0e1-0976-4e93-a7b2-32d9cef09349 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:51.947147 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8873870f-5f4f-47bc-8401-e80bd5e8221f None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:51.950135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d07ab-2650-45ec-ba25-616e3220d87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:51.950135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d07ab-2650-45ec-ba25-616e3220d87a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.950369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d07ab-2650-45ec-ba25-616e3220d87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.950620 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d07ab-2650-45ec-ba25-616e3220d87a admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.951296 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970d07ab-2650-45ec-ba25-616e3220d87a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:01:51.951600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970d07ab-2650-45ec-ba25-616e3220d87a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:51.952152 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8873870f-5f4f-47bc-8401-e80bd5e8221f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:51.954491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970d07ab-2650-45ec-ba25-616e3220d87a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:51.963312 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 775/1549] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:51.968455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 775/1550] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:01:51.969758 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d11f52-60fc-4be1-8149-a83829bed839 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.970046 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d11f52-60fc-4be1-8149-a83829bed839 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.970322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d11f52-60fc-4be1-8149-a83829bed839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.971998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53cbbf-da3a-4298-b94c-be0add1d419d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:51.972318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53cbbf-da3a-4298-b94c-be0add1d419d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:51.972608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53cbbf-da3a-4298-b94c-be0add1d419d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:51.993213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ba53cbbf-da3a-4298-b94c-be0add1d419d None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:51.994343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ba53cbbf-da3a-4298-b94c-be0add1d419d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:52.000990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d2d11f52-60fc-4be1-8149-a83829bed839 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:52.002171 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d2d11f52-60fc-4be1-8149-a83829bed839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:52.008812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 776/1551] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:52.008812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53cbbf-da3a-4298-b94c-be0add1d419d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.018844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53cbbf-da3a-4298-b94c-be0add1d419d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.019583 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 776/1552] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:52.021009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93102ee6-cb1b-4889-aa21-69d09514f91b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.021305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93102ee6-cb1b-4889-aa21-69d09514f91b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.021585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93102ee6-cb1b-4889-aa21-69d09514f91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.024523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d56f5b-db0a-4b97-8c22-2e1d85a3f759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.025243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d56f5b-db0a-4b97-8c22-2e1d85a3f759 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.025531 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d56f5b-db0a-4b97-8c22-2e1d85a3f759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.025808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d56f5b-db0a-4b97-8c22-2e1d85a3f759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.026543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d56f5b-db0a-4b97-8c22-2e1d85a3f759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.028330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d56f5b-db0a-4b97-8c22-2e1d85a3f759 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.039374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 777/1553] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:52.040125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-93102ee6-cb1b-4889-aa21-69d09514f91b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:52.040646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391abc62-0e0a-4cbc-b18a-0ecb88eabf7c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.040933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391abc62-0e0a-4cbc-b18a-0ecb88eabf7c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.041211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391abc62-0e0a-4cbc-b18a-0ecb88eabf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.041558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-93102ee6-cb1b-4889-aa21-69d09514f91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:52.049586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 777/1554] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:52.050874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b611501-8a94-410e-bb8e-62d31a3dbad4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.051162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b611501-8a94-410e-bb8e-62d31a3dbad4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.051433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b611501-8a94-410e-bb8e-62d31a3dbad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.059069 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-391abc62-0e0a-4cbc-b18a-0ecb88eabf7c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:52.060303 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-391abc62-0e0a-4cbc-b18a-0ecb88eabf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:52.068355 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 778/1555] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:52.068914 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391abc62-0e0a-4cbc-b18a-0ecb88eabf7c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.070164 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5b611501-8a94-410e-bb8e-62d31a3dbad4 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:52.071320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5b611501-8a94-410e-bb8e-62d31a3dbad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:52.080316 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 778/1556] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:52.080765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391abc62-0e0a-4cbc-b18a-0ecb88eabf7c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.081552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edaa75-c88f-4b48-bd30-be3ea7c4774b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.081838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edaa75-c88f-4b48-bd30-be3ea7c4774b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.082114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edaa75-c88f-4b48-bd30-be3ea7c4774b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.085548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355a338-fa78-4e94-b489-dfd42578c019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.086209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355a338-fa78-4e94-b489-dfd42578c019 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.086492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355a338-fa78-4e94-b489-dfd42578c019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.086768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355a338-fa78-4e94-b489-dfd42578c019 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.087455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355a338-fa78-4e94-b489-dfd42578c019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.089390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355a338-fa78-4e94-b489-dfd42578c019 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.109623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f3edaa75-c88f-4b48-bd30-be3ea7c4774b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:52.111627 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f3edaa75-c88f-4b48-bd30-be3ea7c4774b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:52.121733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 779/1557] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:52.122988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9550ec-0642-4933-ac4f-47264e8afbbd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.123276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9550ec-0642-4933-ac4f-47264e8afbbd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.123560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9550ec-0642-4933-ac4f-47264e8afbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.130311 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 779/1558] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:52.130311 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355a338-fa78-4e94-b489-dfd42578c019 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.142097 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9b9550ec-0642-4933-ac4f-47264e8afbbd None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:52.143291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9b9550ec-0642-4933-ac4f-47264e8afbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:52.192759 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355a338-fa78-4e94-b489-dfd42578c019 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.197570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b83ff-1cd0-4fbb-9aa5-44b11f28830b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.198234 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b83ff-1cd0-4fbb-9aa5-44b11f28830b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.198520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b83ff-1cd0-4fbb-9aa5-44b11f28830b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.198796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b83ff-1cd0-4fbb-9aa5-44b11f28830b admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.199525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b83ff-1cd0-4fbb-9aa5-44b11f28830b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:01:52.199864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b83ff-1cd0-4fbb-9aa5-44b11f28830b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.200872 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-3c5b83ff-1cd0-4fbb-9aa5-44b11f28830b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:01:52.200872 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 23 11:01:52.200872 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: Jun 23 11:01:52.201765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b83ff-1cd0-4fbb-9aa5-44b11f28830b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.202644 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 780/1559] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:52.203542 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9550ec-0642-4933-ac4f-47264e8afbbd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.204934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:52.221784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 780/1560] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:01:52.221982 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9550ec-0642-4933-ac4f-47264e8afbbd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.222082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b83ff-1cd0-4fbb-9aa5-44b11f28830b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.226592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62350261-c2a9-4dcc-84b8-ffaabd03e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.227264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62350261-c2a9-4dcc-84b8-ffaabd03e782 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.227551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62350261-c2a9-4dcc-84b8-ffaabd03e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.227872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62350261-c2a9-4dcc-84b8-ffaabd03e782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.228603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62350261-c2a9-4dcc-84b8-ffaabd03e782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.228839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b83ff-1cd0-4fbb-9aa5-44b11f28830b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.230240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62350261-c2a9-4dcc-84b8-ffaabd03e782 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.233436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09668c94-4eef-4c30-a3ee-28c3ee9bebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.234074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09668c94-4eef-4c30-a3ee-28c3ee9bebbd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.234356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09668c94-4eef-4c30-a3ee-28c3ee9bebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.234638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09668c94-4eef-4c30-a3ee-28c3ee9bebbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.235467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09668c94-4eef-4c30-a3ee-28c3ee9bebbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:01:52.235817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09668c94-4eef-4c30-a3ee-28c3ee9bebbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.237378 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09668c94-4eef-4c30-a3ee-28c3ee9bebbd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.244044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 781/1561] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:52.244044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc054013-bc0e-4bbc-9cd4-fec141fd3d17 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.244044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc054013-bc0e-4bbc-9cd4-fec141fd3d17 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.244044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc054013-bc0e-4bbc-9cd4-fec141fd3d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.251699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 781/1562] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:01:52.251874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09668c94-4eef-4c30-a3ee-28c3ee9bebbd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.261012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09668c94-4eef-4c30-a3ee-28c3ee9bebbd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.265504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbe7ab-a646-4811-a52e-23f62ddc65b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.266129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbe7ab-a646-4811-a52e-23f62ddc65b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.266406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbe7ab-a646-4811-a52e-23f62ddc65b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.266661 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fc054013-bc0e-4bbc-9cd4-fec141fd3d17 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:52.266778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbe7ab-a646-4811-a52e-23f62ddc65b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.267374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbe7ab-a646-4811-a52e-23f62ddc65b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:01:52.267718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbe7ab-a646-4811-a52e-23f62ddc65b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.267890 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fc054013-bc0e-4bbc-9cd4-fec141fd3d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:52.269302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbe7ab-a646-4811-a52e-23f62ddc65b8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.276274 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 782/1563] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:52.277116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc054013-bc0e-4bbc-9cd4-fec141fd3d17 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.280810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 782/1564] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:01:52.281906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbe7ab-a646-4811-a52e-23f62ddc65b8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.287231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc054013-bc0e-4bbc-9cd4-fec141fd3d17 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.292403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d2bde-9448-4bef-99d9-f3ae54f0b4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.292923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbe7ab-a646-4811-a52e-23f62ddc65b8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.293049 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d2bde-9448-4bef-99d9-f3ae54f0b4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.293321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d2bde-9448-4bef-99d9-f3ae54f0b4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.293594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d2bde-9448-4bef-99d9-f3ae54f0b4c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.294319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d2bde-9448-4bef-99d9-f3ae54f0b4c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:01:52.294631 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d2bde-9448-4bef-99d9-f3ae54f0b4c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.296260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d2bde-9448-4bef-99d9-f3ae54f0b4c4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.297867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5bdc8-4324-40c0-9228-13d6297c5f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.298516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5bdc8-4324-40c0-9228-13d6297c5f0e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.298804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5bdc8-4324-40c0-9228-13d6297c5f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.299117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5bdc8-4324-40c0-9228-13d6297c5f0e admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.299847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5bdc8-4324-40c0-9228-13d6297c5f0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.301850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5bdc8-4324-40c0-9228-13d6297c5f0e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.302855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:52.309052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 783/1565] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:01:52.310270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe1a9f-5714-4813-baf4-bcbf365a4532 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.310553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe1a9f-5714-4813-baf4-bcbf365a4532 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.310830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe1a9f-5714-4813-baf4-bcbf365a4532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.327471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 783/1566] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:52.328230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5bdc8-4324-40c0-9228-13d6297c5f0e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.328990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4bfe1a9f-5714-4813-baf4-bcbf365a4532 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:52.330405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4bfe1a9f-5714-4813-baf4-bcbf365a4532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:52.337953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5bdc8-4324-40c0-9228-13d6297c5f0e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.338116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 784/1567] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:52.338815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe1a9f-5714-4813-baf4-bcbf365a4532 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.342862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9df90f-01da-45b6-8d07-31f751b69365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.343513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9df90f-01da-45b6-8d07-31f751b69365 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.343857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9df90f-01da-45b6-8d07-31f751b69365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.344140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9df90f-01da-45b6-8d07-31f751b69365 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.344829 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9df90f-01da-45b6-8d07-31f751b69365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.346733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9df90f-01da-45b6-8d07-31f751b69365 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.348024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe1a9f-5714-4813-baf4-bcbf365a4532 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.353033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee2298-f69a-4524-a39c-8f414c8d2122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.353889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee2298-f69a-4524-a39c-8f414c8d2122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.354175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee2298-f69a-4524-a39c-8f414c8d2122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.354451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee2298-f69a-4524-a39c-8f414c8d2122 admin admin] PATH_INFO: `/v3/projects/82a06041149a40a6807070f31cf7225e/users/800e9640bdab42bb9a8ea58eecc71653/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.369924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee2298-f69a-4524-a39c-8f414c8d2122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82a06041149a40a6807070f31cf7225e, user_id=800e9640bdab42bb9a8ea58eecc71653, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.371323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-5eee2298-f69a-4524-a39c-8f414c8d2122 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:01:52.371323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 23 11:01:52.371323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: Jun 23 11:01:52.373353 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee2298-f69a-4524-a39c-8f414c8d2122 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.374753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 784/1568] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:52.378309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9df90f-01da-45b6-8d07-31f751b69365 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.396491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 785/1569] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:52 2022] PUT /identity/v3/projects/82a06041149a40a6807070f31cf7225e/users/800e9640bdab42bb9a8ea58eecc71653/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:52.396491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee2298-f69a-4524-a39c-8f414c8d2122 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.437732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9df90f-01da-45b6-8d07-31f751b69365 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.447849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee2298-f69a-4524-a39c-8f414c8d2122 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.452541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c1202-ab7b-4d95-ad10-631da15a61d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.453204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c1202-ab7b-4d95-ad10-631da15a61d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.453487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c1202-ab7b-4d95-ad10-631da15a61d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.453768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c1202-ab7b-4d95-ad10-631da15a61d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.454497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c1202-ab7b-4d95-ad10-631da15a61d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:01:52.454808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c1202-ab7b-4d95-ad10-631da15a61d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.456469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c1202-ab7b-4d95-ad10-631da15a61d3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.468193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 786/1570] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:01:52.468965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c1202-ab7b-4d95-ad10-631da15a61d3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.478792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c1202-ab7b-4d95-ad10-631da15a61d3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.483311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ae66f-c31e-41c3-b647-eb2760ad4c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.483961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ae66f-c31e-41c3-b647-eb2760ad4c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.484252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ae66f-c31e-41c3-b647-eb2760ad4c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.484533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ae66f-c31e-41c3-b647-eb2760ad4c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.485218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ae66f-c31e-41c3-b647-eb2760ad4c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.486224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-418ae66f-c31e-41c3-b647-eb2760ad4c3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:01:52.486224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 23 11:01:52.486224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: Jun 23 11:01:52.487403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ae66f-c31e-41c3-b647-eb2760ad4c3f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.494964 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33283097-6463-4b4c-aab5-a8143f5f4443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.501047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33283097-6463-4b4c-aab5-a8143f5f4443 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.501047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33283097-6463-4b4c-aab5-a8143f5f4443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.501047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33283097-6463-4b4c-aab5-a8143f5f4443 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.501047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33283097-6463-4b4c-aab5-a8143f5f4443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.501047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33283097-6463-4b4c-aab5-a8143f5f4443 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.517817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 787/1571] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:52.517817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ae66f-c31e-41c3-b647-eb2760ad4c3f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.533438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 785/1572] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:52.533438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33283097-6463-4b4c-aab5-a8143f5f4443 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.557101 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:52.578809 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ae66f-c31e-41c3-b647-eb2760ad4c3f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.595779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33283097-6463-4b4c-aab5-a8143f5f4443 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.600366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1341c85-4880-41c4-b577-ea38bc807b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.601699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1341c85-4880-41c4-b577-ea38bc807b59 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.601896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1341c85-4880-41c4-b577-ea38bc807b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.602156 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1341c85-4880-41c4-b577-ea38bc807b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.602881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1341c85-4880-41c4-b577-ea38bc807b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.604516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1341c85-4880-41c4-b577-ea38bc807b59 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.614596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 786/1573] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:52.615403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1341c85-4880-41c4-b577-ea38bc807b59 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.624567 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:52.627368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1341c85-4880-41c4-b577-ea38bc807b59 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.632682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809b714-2bdb-43b9-9cfa-ec86fd71f4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.633325 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809b714-2bdb-43b9-9cfa-ec86fd71f4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.633604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809b714-2bdb-43b9-9cfa-ec86fd71f4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.633879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809b714-2bdb-43b9-9cfa-ec86fd71f4f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.634597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809b714-2bdb-43b9-9cfa-ec86fd71f4f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:01:52.634904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809b714-2bdb-43b9-9cfa-ec86fd71f4f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.636501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809b714-2bdb-43b9-9cfa-ec86fd71f4f3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.638410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1633c1-96ac-4a87-a529-cc16ec4653df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.639148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1633c1-96ac-4a87-a529-cc16ec4653df admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.639461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1633c1-96ac-4a87-a529-cc16ec4653df admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.639778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1633c1-96ac-4a87-a529-cc16ec4653df admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.640480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1633c1-96ac-4a87-a529-cc16ec4653df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.642433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1633c1-96ac-4a87-a529-cc16ec4653df admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.648371 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 787/1574] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:01:52.649136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809b714-2bdb-43b9-9cfa-ec86fd71f4f3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.660225 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809b714-2bdb-43b9-9cfa-ec86fd71f4f3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.665490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13acc3e-9304-4ee9-9680-a676039b0cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.666133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13acc3e-9304-4ee9-9680-a676039b0cea admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.666418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13acc3e-9304-4ee9-9680-a676039b0cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.666696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13acc3e-9304-4ee9-9680-a676039b0cea admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.667371 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13acc3e-9304-4ee9-9680-a676039b0cea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.669298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13acc3e-9304-4ee9-9680-a676039b0cea admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.671302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 788/1575] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/projects => generated 379 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:52.672514 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec4810-f419-4d54-9ef7-ef413ecfbfb7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.672808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec4810-f419-4d54-9ef7-ef413ecfbfb7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.673118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec4810-f419-4d54-9ef7-ef413ecfbfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.689338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 788/1576] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/users => generated 283 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:52.692547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13acc3e-9304-4ee9-9680-a676039b0cea admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.699411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9fec4810-f419-4d54-9ef7-ef413ecfbfb7 None None] MFA Rules not processed for user `800e9640bdab42bb9a8ea58eecc71653`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:52.700687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9fec4810-f419-4d54-9ef7-ef413ecfbfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:52.702363 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13acc3e-9304-4ee9-9680-a676039b0cea admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.759613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53090989-ef15-4c8d-b8fc-8c71adde6e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.760222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53090989-ef15-4c8d-b8fc-8c71adde6e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.760508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53090989-ef15-4c8d-b8fc-8c71adde6e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.760788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53090989-ef15-4c8d-b8fc-8c71adde6e56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.761476 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53090989-ef15-4c8d-b8fc-8c71adde6e56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.763640 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53090989-ef15-4c8d-b8fc-8c71adde6e56 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.765133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 789/1577] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:52.765947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec4810-f419-4d54-9ef7-ef413ecfbfb7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.776446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec4810-f419-4d54-9ef7-ef413ecfbfb7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.781013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125adac-a778-44f8-9148-d846d927cce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.782017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125adac-a778-44f8-9148-d846d927cce2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.782017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125adac-a778-44f8-9148-d846d927cce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.782223 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125adac-a778-44f8-9148-d846d927cce2 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.782927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125adac-a778-44f8-9148-d846d927cce2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.784874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125adac-a778-44f8-9148-d846d927cce2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.802908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 789/1578] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:52.803548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53090989-ef15-4c8d-b8fc-8c71adde6e56 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.813093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 790/1579] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:52.813093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125adac-a778-44f8-9148-d846d927cce2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.821399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125adac-a778-44f8-9148-d846d927cce2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.866643 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53090989-ef15-4c8d-b8fc-8c71adde6e56 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.871797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7cb6c-fdd0-4d7b-85da-b349b0f34e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.872463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7cb6c-fdd0-4d7b-85da-b349b0f34e71 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.872744 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7cb6c-fdd0-4d7b-85da-b349b0f34e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.873024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7cb6c-fdd0-4d7b-85da-b349b0f34e71 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.873710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7cb6c-fdd0-4d7b-85da-b349b0f34e71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.875746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7cb6c-fdd0-4d7b-85da-b349b0f34e71 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.878703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8acc23-31ff-481d-9897-9399dea2336b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.879372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8acc23-31ff-481d-9897-9399dea2336b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.879678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8acc23-31ff-481d-9897-9399dea2336b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.879971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8acc23-31ff-481d-9897-9399dea2336b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.880705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8acc23-31ff-481d-9897-9399dea2336b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.882809 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8acc23-31ff-481d-9897-9399dea2336b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.893346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 791/1580] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:52.894215 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8acc23-31ff-481d-9897-9399dea2336b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.899463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 790/1581] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:52.899463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7cb6c-fdd0-4d7b-85da-b349b0f34e71 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.902596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:52.904947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8acc23-31ff-481d-9897-9399dea2336b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.907533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7cb6c-fdd0-4d7b-85da-b349b0f34e71 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.909716 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4e739-3cb8-47a2-b445-6a30d19c4cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.910493 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4e739-3cb8-47a2-b445-6a30d19c4cab admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.910820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4e739-3cb8-47a2-b445-6a30d19c4cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.911104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4e739-3cb8-47a2-b445-6a30d19c4cab admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.911827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4e739-3cb8-47a2-b445-6a30d19c4cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.913765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4e739-3cb8-47a2-b445-6a30d19c4cab admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.914064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d8ed2-c87d-43e5-a052-6c32ee24f442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:52.914717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d8ed2-c87d-43e5-a052-6c32ee24f442 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.915057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d8ed2-c87d-43e5-a052-6c32ee24f442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.915346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d8ed2-c87d-43e5-a052-6c32ee24f442 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.916064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d8ed2-c87d-43e5-a052-6c32ee24f442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:52.917998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d8ed2-c87d-43e5-a052-6c32ee24f442 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:52.938443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 791/1582] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/users => generated 290 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:52.939277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d8ed2-c87d-43e5-a052-6c32ee24f442 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.943687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 792/1583] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:52.945428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fdf4b-661a-434b-9648-9c59294c3dcf None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:52.945724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fdf4b-661a-434b-9648-9c59294c3dcf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:52.946010 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fdf4b-661a-434b-9648-9c59294c3dcf None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:52.947332 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 793/1584] 10.222.0.65 () {54 vars in 961 bytes} [Thu Jun 23 11:01:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:01:52.952619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fdf4b-661a-434b-9648-9c59294c3dcf None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:52.954240 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d8ed2-c87d-43e5-a052-6c32ee24f442 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:52.972757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:53.012565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06d0c3-3b19-41a4-a0fb-54bfda982564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.013229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06d0c3-3b19-41a4-a0fb-54bfda982564 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.013519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06d0c3-3b19-41a4-a0fb-54bfda982564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.013796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06d0c3-3b19-41a4-a0fb-54bfda982564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.014523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06d0c3-3b19-41a4-a0fb-54bfda982564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.016320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06d0c3-3b19-41a4-a0fb-54bfda982564 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.019841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fdf4b-661a-434b-9648-9c59294c3dcf None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.025391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1b7d2-dddc-4ae1-8b48-3b82f2c672f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.026061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1b7d2-dddc-4ae1-8b48-3b82f2c672f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.026358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1b7d2-dddc-4ae1-8b48-3b82f2c672f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.026493 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 792/1585] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:53.026683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1b7d2-dddc-4ae1-8b48-3b82f2c672f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.027209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06d0c3-3b19-41a4-a0fb-54bfda982564 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.027416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1b7d2-dddc-4ae1-8b48-3b82f2c672f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.029703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1b7d2-dddc-4ae1-8b48-3b82f2c672f0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.038419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06d0c3-3b19-41a4-a0fb-54bfda982564 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.043687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3d14d-a395-4ff2-883a-a6bedcc94314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.044539 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3d14d-a395-4ff2-883a-a6bedcc94314 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.044832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3d14d-a395-4ff2-883a-a6bedcc94314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.045117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3d14d-a395-4ff2-883a-a6bedcc94314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.045840 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3d14d-a395-4ff2-883a-a6bedcc94314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.047422 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3d14d-a395-4ff2-883a-a6bedcc94314 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.052946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 794/1586] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:01:52 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:53.052946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1b7d2-dddc-4ae1-8b48-3b82f2c672f0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.061121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 793/1587] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:53.061121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3d14d-a395-4ff2-883a-a6bedcc94314 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.061458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1b7d2-dddc-4ae1-8b48-3b82f2c672f0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.064570 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10cb4d-a0ea-4124-a091-75d2815cfb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.065439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10cb4d-a0ea-4124-a091-75d2815cfb47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.065728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10cb4d-a0ea-4124-a091-75d2815cfb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.066011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10cb4d-a0ea-4124-a091-75d2815cfb47 admin admin] PATH_INFO: `/v3/projects/ccf2ef30c6f84aafb066cfa08c397d05/users/f55347c643984fb6a4a8655ad992862d/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.069085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3d14d-a395-4ff2-883a-a6bedcc94314 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.074192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e1fe1-431c-4fad-877f-7e45fc19aa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.074835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e1fe1-431c-4fad-877f-7e45fc19aa29 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.075120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e1fe1-431c-4fad-877f-7e45fc19aa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.075401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e1fe1-431c-4fad-877f-7e45fc19aa29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.076157 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e1fe1-431c-4fad-877f-7e45fc19aa29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.077736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e1fe1-431c-4fad-877f-7e45fc19aa29 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.082105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10cb4d-a0ea-4124-a091-75d2815cfb47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf2ef30c6f84aafb066cfa08c397d05, user_id=f55347c643984fb6a4a8655ad992862d, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.085880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10cb4d-a0ea-4124-a091-75d2815cfb47 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.092477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 794/1588] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:53.092477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e1fe1-431c-4fad-877f-7e45fc19aa29 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.102188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e1fe1-431c-4fad-877f-7e45fc19aa29 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.104304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 795/1589] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:53 2022] PUT /identity/v3/projects/ccf2ef30c6f84aafb066cfa08c397d05/users/f55347c643984fb6a4a8655ad992862d/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:53.105292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10cb4d-a0ea-4124-a091-75d2815cfb47 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.159300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b268a-44a6-4000-bc84-f35ad9dd6014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.160004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b268a-44a6-4000-bc84-f35ad9dd6014 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.160290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b268a-44a6-4000-bc84-f35ad9dd6014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.160568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b268a-44a6-4000-bc84-f35ad9dd6014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.161297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b268a-44a6-4000-bc84-f35ad9dd6014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.162901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b268a-44a6-4000-bc84-f35ad9dd6014 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.172598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 795/1590] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:53.173958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e32939-b315-4d0d-aedc-45a46e94ac26 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.174257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e32939-b315-4d0d-aedc-45a46e94ac26 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.174538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e32939-b315-4d0d-aedc-45a46e94ac26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.177762 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10cb4d-a0ea-4124-a091-75d2815cfb47 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.183099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a9c18-54f0-459a-bcd7-969871540612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.183812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a9c18-54f0-459a-bcd7-969871540612 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.184099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a9c18-54f0-459a-bcd7-969871540612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.184378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a9c18-54f0-459a-bcd7-969871540612 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.185099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a9c18-54f0-459a-bcd7-969871540612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.187015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a9c18-54f0-459a-bcd7-969871540612 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.192680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a5e32939-b315-4d0d-aedc-45a46e94ac26 None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:53.193909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a5e32939-b315-4d0d-aedc-45a46e94ac26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:53.210170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 796/1591] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:01:53 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:53.210170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a9c18-54f0-459a-bcd7-969871540612 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.217433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a9c18-54f0-459a-bcd7-969871540612 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.222006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f68813-8bfb-408e-b95e-ead55b4df511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.222651 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f68813-8bfb-408e-b95e-ead55b4df511 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.222930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f68813-8bfb-408e-b95e-ead55b4df511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.223206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f68813-8bfb-408e-b95e-ead55b4df511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.223950 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f68813-8bfb-408e-b95e-ead55b4df511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.225520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f68813-8bfb-408e-b95e-ead55b4df511 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.235246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 797/1592] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:53.236050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f68813-8bfb-408e-b95e-ead55b4df511 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.246198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f68813-8bfb-408e-b95e-ead55b4df511 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.249684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:53.251450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d62a5c-a695-496a-a4a8-3ba57e027f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.252212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d62a5c-a695-496a-a4a8-3ba57e027f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.252468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d62a5c-a695-496a-a4a8-3ba57e027f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.252745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d62a5c-a695-496a-a4a8-3ba57e027f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.253467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d62a5c-a695-496a-a4a8-3ba57e027f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.255140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d62a5c-a695-496a-a4a8-3ba57e027f53 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.266327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 798/1593] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:53.267137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d62a5c-a695-496a-a4a8-3ba57e027f53 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.270659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 796/1594] 10.222.0.65 () {58 vars in 1058 bytes} [Thu Jun 23 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:53.271476 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e32939-b315-4d0d-aedc-45a46e94ac26 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.278138 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d62a5c-a695-496a-a4a8-3ba57e027f53 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.281842 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e32939-b315-4d0d-aedc-45a46e94ac26 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.282799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a5634-9131-45c1-822a-6069e782b6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.283441 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a5634-9131-45c1-822a-6069e782b6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.283762 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a5634-9131-45c1-822a-6069e782b6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.284043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a5634-9131-45c1-822a-6069e782b6b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.284765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a5634-9131-45c1-822a-6069e782b6b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.286398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a5634-9131-45c1-822a-6069e782b6b6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.286649 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcda922-dd64-4701-8dbc-a69650ff3f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.287287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcda922-dd64-4701-8dbc-a69650ff3f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.287575 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcda922-dd64-4701-8dbc-a69650ff3f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.287885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcda922-dd64-4701-8dbc-a69650ff3f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.288645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcda922-dd64-4701-8dbc-a69650ff3f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.290214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcda922-dd64-4701-8dbc-a69650ff3f71 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.296794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 799/1595] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:53.298110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2edf9-cdd1-422a-ba76-592882bd47e4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.298398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2edf9-cdd1-422a-ba76-592882bd47e4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.298682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2edf9-cdd1-422a-ba76-592882bd47e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.307035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 797/1596] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:53.307035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcda922-dd64-4701-8dbc-a69650ff3f71 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.314433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcda922-dd64-4701-8dbc-a69650ff3f71 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.318990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925179a3-a305-4201-8d21-6a42c40b1b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.320201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925179a3-a305-4201-8d21-6a42c40b1b9a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.320489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925179a3-a305-4201-8d21-6a42c40b1b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.320762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925179a3-a305-4201-8d21-6a42c40b1b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.321475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925179a3-a305-4201-8d21-6a42c40b1b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.323033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925179a3-a305-4201-8d21-6a42c40b1b9a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.325483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d2d2edf9-cdd1-422a-ba76-592882bd47e4 None None] MFA Rules not processed for user `f55347c643984fb6a4a8655ad992862d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:53.326686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d2d2edf9-cdd1-422a-ba76-592882bd47e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:53.332800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 798/1597] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:53.333582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925179a3-a305-4201-8d21-6a42c40b1b9a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.344349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925179a3-a305-4201-8d21-6a42c40b1b9a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.348852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80077bab-cff4-43d7-adec-3838df84b514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.349509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80077bab-cff4-43d7-adec-3838df84b514 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.349792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80077bab-cff4-43d7-adec-3838df84b514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.350188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80077bab-cff4-43d7-adec-3838df84b514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.350920 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80077bab-cff4-43d7-adec-3838df84b514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.352499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80077bab-cff4-43d7-adec-3838df84b514 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.355643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:53.363624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 799/1598] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:53.364558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80077bab-cff4-43d7-adec-3838df84b514 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.375052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80077bab-cff4-43d7-adec-3838df84b514 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.379600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497eaf8-ff16-4c08-9806-91ad5432a9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.380475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497eaf8-ff16-4c08-9806-91ad5432a9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.380755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497eaf8-ff16-4c08-9806-91ad5432a9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.381034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497eaf8-ff16-4c08-9806-91ad5432a9c6 admin admin] PATH_INFO: `/v3/projects/b7156540914b4b949c71dfc09f71e6ec/users/6d1188e9e13642449f694f50528f2723/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.393613 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 800/1599] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:53.394558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2edf9-cdd1-422a-ba76-592882bd47e4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.396143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497eaf8-ff16-4c08-9806-91ad5432a9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7156540914b4b949c71dfc09f71e6ec, user_id=6d1188e9e13642449f694f50528f2723, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.398851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497eaf8-ff16-4c08-9806-91ad5432a9c6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.405425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2edf9-cdd1-422a-ba76-592882bd47e4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.410523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1d27b-159b-4a45-bbce-848059a855f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.411197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1d27b-159b-4a45-bbce-848059a855f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.411477 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1d27b-159b-4a45-bbce-848059a855f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.411787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1d27b-159b-4a45-bbce-848059a855f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.412524 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1d27b-159b-4a45-bbce-848059a855f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.413512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 800/1600] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:53 2022] PUT /identity/v3/projects/b7156540914b4b949c71dfc09f71e6ec/users/6d1188e9e13642449f694f50528f2723/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:53.414132 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1d27b-159b-4a45-bbce-848059a855f3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.414419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497eaf8-ff16-4c08-9806-91ad5432a9c6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.427485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 801/1601] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:53.427485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1d27b-159b-4a45-bbce-848059a855f3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.436019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1d27b-159b-4a45-bbce-848059a855f3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.491273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497eaf8-ff16-4c08-9806-91ad5432a9c6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.492535 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3d267-c968-4950-8043-c8dc5ff86dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.493445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3d267-c968-4950-8043-c8dc5ff86dba admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.493753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3d267-c968-4950-8043-c8dc5ff86dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.494030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3d267-c968-4950-8043-c8dc5ff86dba admin admin] PATH_INFO: `/v3/projects/d3db3ac9cb13453fa7bc8195bdf88fd1/users/6bb5a1f18dff4395bec9c205a9937493/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.497987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdd6c9-daf9-4852-9163-3f5d678f8271 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.498640 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdd6c9-daf9-4852-9163-3f5d678f8271 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.498928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdd6c9-daf9-4852-9163-3f5d678f8271 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.499206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdd6c9-daf9-4852-9163-3f5d678f8271 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.511561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3d267-c968-4950-8043-c8dc5ff86dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3db3ac9cb13453fa7bc8195bdf88fd1, user_id=6bb5a1f18dff4395bec9c205a9937493, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.513395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3d267-c968-4950-8043-c8dc5ff86dba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.528869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 802/1602] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:53 2022] PUT /identity/v3/projects/d3db3ac9cb13453fa7bc8195bdf88fd1/users/6bb5a1f18dff4395bec9c205a9937493/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:53.529675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3d267-c968-4950-8043-c8dc5ff86dba admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.559068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdd6c9-daf9-4852-9163-3f5d678f8271 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.560691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdd6c9-daf9-4852-9163-3f5d678f8271 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.569643 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 801/1603] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:53.570310 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdd6c9-daf9-4852-9163-3f5d678f8271 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.580275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdd6c9-daf9-4852-9163-3f5d678f8271 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.591377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3d267-c968-4950-8043-c8dc5ff86dba admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.596337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e7357-cf2e-4412-b783-b60faa94073f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.597231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e7357-cf2e-4412-b783-b60faa94073f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.597511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e7357-cf2e-4412-b783-b60faa94073f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.597790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e7357-cf2e-4412-b783-b60faa94073f admin admin] PATH_INFO: `/v3/projects/ac9f9313c3ac4dde97e75ac6fa8fae11/users/303e8aae1bd04228bedb145dec891a3e/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.606011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:53.619639 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e7357-cf2e-4412-b783-b60faa94073f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f9313c3ac4dde97e75ac6fa8fae11, user_id=303e8aae1bd04228bedb145dec891a3e, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.622703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e7357-cf2e-4412-b783-b60faa94073f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.640438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 803/1604] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:53 2022] PUT /identity/v3/projects/ac9f9313c3ac4dde97e75ac6fa8fae11/users/303e8aae1bd04228bedb145dec891a3e/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:53.640438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e7357-cf2e-4412-b783-b60faa94073f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.652461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63623303-2c97-4668-b739-9251ea7c0e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.653361 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63623303-2c97-4668-b739-9251ea7c0e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.653754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63623303-2c97-4668-b739-9251ea7c0e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.654039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63623303-2c97-4668-b739-9251ea7c0e54 admin admin] PATH_INFO: `/v3/projects/c6c6487f3e094808b1665bdd93da9b3c/users/f6390dc942db430e8dd648901900432b/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.667965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:53.671601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63623303-2c97-4668-b739-9251ea7c0e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c6487f3e094808b1665bdd93da9b3c, user_id=f6390dc942db430e8dd648901900432b, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.674560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63623303-2c97-4668-b739-9251ea7c0e54 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.689114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 802/1605] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:53 2022] PUT /identity/v3/projects/c6c6487f3e094808b1665bdd93da9b3c/users/f6390dc942db430e8dd648901900432b/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:53.690008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63623303-2c97-4668-b739-9251ea7c0e54 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.713923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e7357-cf2e-4412-b783-b60faa94073f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.755274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63623303-2c97-4668-b739-9251ea7c0e54 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.760037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f956f5-cd5d-432e-add9-49f0b344a004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.760696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f956f5-cd5d-432e-add9-49f0b344a004 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.761102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f956f5-cd5d-432e-add9-49f0b344a004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.761384 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f956f5-cd5d-432e-add9-49f0b344a004 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.762222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f956f5-cd5d-432e-add9-49f0b344a004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.764151 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f956f5-cd5d-432e-add9-49f0b344a004 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.777490 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4697f3-613c-40ec-bbb6-4ce263189b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.778345 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4697f3-613c-40ec-bbb6-4ce263189b3f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.778486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4697f3-613c-40ec-bbb6-4ce263189b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.778709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4697f3-613c-40ec-bbb6-4ce263189b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.779521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4697f3-613c-40ec-bbb6-4ce263189b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.781163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4697f3-613c-40ec-bbb6-4ce263189b3f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.797577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 803/1606] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:01:53 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:53.797577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631c960-91a3-453b-8fd5-46f20a33530a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.797577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631c960-91a3-453b-8fd5-46f20a33530a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.797577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631c960-91a3-453b-8fd5-46f20a33530a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.799712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 804/1607] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:53.800611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4697f3-613c-40ec-bbb6-4ce263189b3f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.811333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4697f3-613c-40ec-bbb6-4ce263189b3f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.826498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0631c960-91a3-453b-8fd5-46f20a33530a None None] MFA Rules not processed for user `6d1188e9e13642449f694f50528f2723`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:53.827857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0631c960-91a3-453b-8fd5-46f20a33530a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:53.874614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d38f28-e908-4914-b14d-cce7c7a3a785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.875510 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d38f28-e908-4914-b14d-cce7c7a3a785 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.875853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d38f28-e908-4914-b14d-cce7c7a3a785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.876141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d38f28-e908-4914-b14d-cce7c7a3a785 admin admin] PATH_INFO: `/v3/projects/578cf1b29d474080a716e77fe2e84e5b/users/df7d511abd8544878e7d0097a1ab0319/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.891911 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d38f28-e908-4914-b14d-cce7c7a3a785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=578cf1b29d474080a716e77fe2e84e5b, user_id=df7d511abd8544878e7d0097a1ab0319, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.894723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d38f28-e908-4914-b14d-cce7c7a3a785 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.901830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 804/1608] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:53.903843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8f780-320a-41ce-a4a0-d1444a44e94f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.904039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8f780-320a-41ce-a4a0-d1444a44e94f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.904329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8f780-320a-41ce-a4a0-d1444a44e94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.909958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 805/1609] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:53 2022] PUT /identity/v3/projects/578cf1b29d474080a716e77fe2e84e5b/users/df7d511abd8544878e7d0097a1ab0319/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:53.911248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfbabd-29ad-4801-837b-276ab9dd87e1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.911679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfbabd-29ad-4801-837b-276ab9dd87e1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.911980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfbabd-29ad-4801-837b-276ab9dd87e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.927276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ecb8f780-320a-41ce-a4a0-d1444a44e94f None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:53.928534 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ecb8f780-320a-41ce-a4a0-d1444a44e94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:53.929978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0acfbabd-29ad-4801-837b-276ab9dd87e1 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:53.931293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0acfbabd-29ad-4801-837b-276ab9dd87e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:53.948155 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:53.981233 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:54.002970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 806/1610] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:54.002970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1433a1-1516-45b9-bf63-841966359dfd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.002970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1433a1-1516-45b9-bf63-841966359dfd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.002970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1433a1-1516-45b9-bf63-841966359dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.022359 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6b1433a1-1516-45b9-bf63-841966359dfd None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:54.024271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6b1433a1-1516-45b9-bf63-841966359dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:54.045962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 807/1611] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:54.045962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1433a1-1516-45b9-bf63-841966359dfd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.067500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1433a1-1516-45b9-bf63-841966359dfd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.072593 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263383e-29d4-4984-9e82-1b3014e44b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.073495 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263383e-29d4-4984-9e82-1b3014e44b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.073776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263383e-29d4-4984-9e82-1b3014e44b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.074052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263383e-29d4-4984-9e82-1b3014e44b4a admin admin] PATH_INFO: `/v3/projects/06e49e2ab21d4f82b2b10dd49330a703/users/ac6a4ac364db438494a8374781ce4ac3/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.094121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263383e-29d4-4984-9e82-1b3014e44b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e49e2ab21d4f82b2b10dd49330a703, user_id=ac6a4ac364db438494a8374781ce4ac3, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.097627 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263383e-29d4-4984-9e82-1b3014e44b4a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.114591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 808/1612] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:54 2022] PUT /identity/v3/projects/06e49e2ab21d4f82b2b10dd49330a703/users/ac6a4ac364db438494a8374781ce4ac3/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:54.115721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263383e-29d4-4984-9e82-1b3014e44b4a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.126994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 805/1613] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:54.127729 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8f780-320a-41ce-a4a0-d1444a44e94f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.137079 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8f780-320a-41ce-a4a0-d1444a44e94f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.180521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263383e-29d4-4984-9e82-1b3014e44b4a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.185343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734dbe2-1530-4537-ac32-e403e517977d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.186014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734dbe2-1530-4537-ac32-e403e517977d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.186295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734dbe2-1530-4537-ac32-e403e517977d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.186571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734dbe2-1530-4537-ac32-e403e517977d admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.187357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734dbe2-1530-4537-ac32-e403e517977d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.189321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734dbe2-1530-4537-ac32-e403e517977d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.193840 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cbbb6-ba1a-497e-b979-ab789ab3b8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.194500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cbbb6-ba1a-497e-b979-ab789ab3b8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.194975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cbbb6-ba1a-497e-b979-ab789ab3b8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.195267 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cbbb6-ba1a-497e-b979-ab789ab3b8b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.195993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cbbb6-ba1a-497e-b979-ab789ab3b8b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.197873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cbbb6-ba1a-497e-b979-ab789ab3b8b8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.208428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 809/1614] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:01:54 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:54.209890 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d279b8-69b9-417b-b651-e066e7a31c03 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.210775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d279b8-69b9-417b-b651-e066e7a31c03 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.211148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d279b8-69b9-417b-b651-e066e7a31c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.230193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 806/1615] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:01:54 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:54.230193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cbbb6-ba1a-497e-b979-ab789ab3b8b8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.230487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-88d279b8-69b9-417b-b651-e066e7a31c03 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:54.230586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ca7cbbb6-ba1a-497e-b979-ab789ab3b8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:54.230685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-88d279b8-69b9-417b-b651-e066e7a31c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:54.285671 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cbbb6-ba1a-497e-b979-ab789ab3b8b8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.289794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011bb84-d4b5-4281-a2a9-8e7406ddf8d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.290512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 810/1616] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:54.290629 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011bb84-d4b5-4281-a2a9-8e7406ddf8d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.290794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011bb84-d4b5-4281-a2a9-8e7406ddf8d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.291076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011bb84-d4b5-4281-a2a9-8e7406ddf8d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.291349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d279b8-69b9-417b-b651-e066e7a31c03 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.300385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d279b8-69b9-417b-b651-e066e7a31c03 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.304040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4464c-81a5-4c33-96c0-7222aceb3a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.304287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011bb84-d4b5-4281-a2a9-8e7406ddf8d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.304708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4464c-81a5-4c33-96c0-7222aceb3a2b service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.304993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4464c-81a5-4c33-96c0-7222aceb3a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.305270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4464c-81a5-4c33-96c0-7222aceb3a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.305849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011bb84-d4b5-4281-a2a9-8e7406ddf8d6 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.314335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4464c-81a5-4c33-96c0-7222aceb3a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.315832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 807/1617] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:54.315965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4464c-81a5-4c33-96c0-7222aceb3a2b service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.316978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e430d0a-5bed-4660-853f-f538e527b437 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.317233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e430d0a-5bed-4660-853f-f538e527b437 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.317513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e430d0a-5bed-4660-853f-f538e527b437 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.318894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 808/1618] 10.222.0.65 () {54 vars in 961 bytes} [Thu Jun 23 11:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:01:54.320338 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceab3ea-2da0-4657-ad29-22ef23276186 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.320622 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceab3ea-2da0-4657-ad29-22ef23276186 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.320899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceab3ea-2da0-4657-ad29-22ef23276186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.336895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 811/1619] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:54.337946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4464c-81a5-4c33-96c0-7222aceb3a2b service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.348181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4464c-81a5-4c33-96c0-7222aceb3a2b service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.349337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fceab3ea-2da0-4657-ad29-22ef23276186 None None] MFA Rules not processed for user `ac6a4ac364db438494a8374781ce4ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:54.350609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fceab3ea-2da0-4657-ad29-22ef23276186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:54.352125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317562f-2816-4faa-ace4-4634e66edaf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.353145 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317562f-2816-4faa-ace4-4634e66edaf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.353145 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317562f-2816-4faa-ace4-4634e66edaf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.353398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317562f-2816-4faa-ace4-4634e66edaf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.370470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317562f-2816-4faa-ace4-4634e66edaf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.371963 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317562f-2816-4faa-ace4-4634e66edaf9 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.384570 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 812/1620] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:54.385824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317562f-2816-4faa-ace4-4634e66edaf9 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.398769 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317562f-2816-4faa-ace4-4634e66edaf9 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.405287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f0283-f207-4504-8a72-1e97fe32a28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.406183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:54.407140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f0283-f207-4504-8a72-1e97fe32a28c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.407512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f0283-f207-4504-8a72-1e97fe32a28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.407768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f0283-f207-4504-8a72-1e97fe32a28c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.408805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f0283-f207-4504-8a72-1e97fe32a28c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.411260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f0283-f207-4504-8a72-1e97fe32a28c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.429749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 813/1621] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:54.429749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f0283-f207-4504-8a72-1e97fe32a28c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.437301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 809/1622] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:54.438523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceab3ea-2da0-4657-ad29-22ef23276186 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.444523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f0283-f207-4504-8a72-1e97fe32a28c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.449885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceab3ea-2da0-4657-ad29-22ef23276186 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.453352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6153f74-4d76-49b2-b734-c91404fcffea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.454211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bfafc-8206-42ea-a3ee-e4034187d337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.454792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6153f74-4d76-49b2-b734-c91404fcffea admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.454918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bfafc-8206-42ea-a3ee-e4034187d337 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.455203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bfafc-8206-42ea-a3ee-e4034187d337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.455711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bfafc-8206-42ea-a3ee-e4034187d337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.456214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6153f74-4d76-49b2-b734-c91404fcffea admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.457214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6153f74-4d76-49b2-b734-c91404fcffea admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.458407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6153f74-4d76-49b2-b734-c91404fcffea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.461550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6153f74-4d76-49b2-b734-c91404fcffea admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.468601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2bfafc-8206-42ea-a3ee-e4034187d337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.470356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2bfafc-8206-42ea-a3ee-e4034187d337 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.475344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:54.483426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 810/1623] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:54.484959 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa96c9b-2d40-4e1a-be8f-60afa86a3c71 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.485117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa96c9b-2d40-4e1a-be8f-60afa86a3c71 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.485609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa96c9b-2d40-4e1a-be8f-60afa86a3c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.496011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 814/1624] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:01:54 2022] POST /identity/v3/users => generated 288 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:54.496011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6153f74-4d76-49b2-b734-c91404fcffea admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.505628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6153f74-4d76-49b2-b734-c91404fcffea admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.511397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9aa96c9b-2d40-4e1a-be8f-60afa86a3c71 None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:54.512441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9aa96c9b-2d40-4e1a-be8f-60afa86a3c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:54.513099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030144a-3ece-40b8-908f-9482738ca9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.514110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030144a-3ece-40b8-908f-9482738ca9d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.514613 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030144a-3ece-40b8-908f-9482738ca9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.514990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030144a-3ece-40b8-908f-9482738ca9d2 admin admin] PATH_INFO: `/v3/projects/f12f6358915b44ce8f2891e509214d50/users/f9cb2b5c72f74db9aac85848eb5dcf13/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.532691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030144a-3ece-40b8-908f-9482738ca9d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12f6358915b44ce8f2891e509214d50, user_id=f9cb2b5c72f74db9aac85848eb5dcf13, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.536518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030144a-3ece-40b8-908f-9482738ca9d2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.560800 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 815/1625] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:54 2022] PUT /identity/v3/projects/f12f6358915b44ce8f2891e509214d50/users/f9cb2b5c72f74db9aac85848eb5dcf13/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:54.561751 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030144a-3ece-40b8-908f-9482738ca9d2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.596816 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 811/1626] 10.222.0.65 () {58 vars in 1058 bytes} [Thu Jun 23 11:01:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:54.598610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa96c9b-2d40-4e1a-be8f-60afa86a3c71 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.614758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa96c9b-2d40-4e1a-be8f-60afa86a3c71 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.637924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030144a-3ece-40b8-908f-9482738ca9d2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.646922 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f16a69-de80-4045-905c-00839588690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.648505 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f16a69-de80-4045-905c-00839588690b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.648965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f16a69-de80-4045-905c-00839588690b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.649409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f16a69-de80-4045-905c-00839588690b admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.650282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f16a69-de80-4045-905c-00839588690b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.652703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f16a69-de80-4045-905c-00839588690b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.698013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 816/1627] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:01:54 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:54.702017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f16a69-de80-4045-905c-00839588690b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.745929 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d88d1-f365-4740-b705-3bfa483cc8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.746678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d88d1-f365-4740-b705-3bfa483cc8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.746967 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d88d1-f365-4740-b705-3bfa483cc8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.747445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d88d1-f365-4740-b705-3bfa483cc8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.748565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d88d1-f365-4740-b705-3bfa483cc8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.750647 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d88d1-f365-4740-b705-3bfa483cc8aa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.765276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 812/1628] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:54.767319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3198088-bddb-49a1-b794-1529733e29d7 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.767613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3198088-bddb-49a1-b794-1529733e29d7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.768037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3198088-bddb-49a1-b794-1529733e29d7 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.770708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 813/1629] 10.222.0.65 () {54 vars in 912 bytes} [Thu Jun 23 11:01:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:01:54.772428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3198088-bddb-49a1-b794-1529733e29d7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.809978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f16a69-de80-4045-905c-00839588690b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.816908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64fcc4-acad-4e1d-854c-6fa942d5a91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.818227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64fcc4-acad-4e1d-854c-6fa942d5a91b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.818653 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64fcc4-acad-4e1d-854c-6fa942d5a91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.819533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64fcc4-acad-4e1d-854c-6fa942d5a91b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.821099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64fcc4-acad-4e1d-854c-6fa942d5a91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.823198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64fcc4-acad-4e1d-854c-6fa942d5a91b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.848221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 817/1630] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:54.849396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64fcc4-acad-4e1d-854c-6fa942d5a91b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.862302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64fcc4-acad-4e1d-854c-6fa942d5a91b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.889452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940e5fa-1982-4091-b502-616388da2762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.892824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940e5fa-1982-4091-b502-616388da2762 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.894197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940e5fa-1982-4091-b502-616388da2762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.895189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940e5fa-1982-4091-b502-616388da2762 admin admin] PATH_INFO: `/v3/projects/0e8b6068752a4bee8bd24f49cd9094ca/users/944061fbbf244a688e427239f2710d10/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.904500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:54.917635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3198088-bddb-49a1-b794-1529733e29d7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:54.945430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524009c-94f3-4a6f-ae98-2f12e0bec317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:54.948197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524009c-94f3-4a6f-ae98-2f12e0bec317 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.948861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524009c-94f3-4a6f-ae98-2f12e0bec317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.950020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524009c-94f3-4a6f-ae98-2f12e0bec317 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:54.950841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524009c-94f3-4a6f-ae98-2f12e0bec317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.953386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524009c-94f3-4a6f-ae98-2f12e0bec317 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.954373 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940e5fa-1982-4091-b502-616388da2762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8b6068752a4bee8bd24f49cd9094ca, user_id=944061fbbf244a688e427239f2710d10, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:54.958733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940e5fa-1982-4091-b502-616388da2762 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:54.991046 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 814/1632] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:01:54 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:01:54.991282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 818/1632] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:54 2022] PUT /identity/v3/projects/0e8b6068752a4bee8bd24f49cd9094ca/users/944061fbbf244a688e427239f2710d10/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:54.991282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940e5fa-1982-4091-b502-616388da2762 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:54.992062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c15e5-3ebf-40cf-9345-3baf810d4a5d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:54.992062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c15e5-3ebf-40cf-9345-3baf810d4a5d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:54.992062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c15e5-3ebf-40cf-9345-3baf810d4a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.022131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a55c15e5-3ebf-40cf-9345-3baf810d4a5d None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:55.024134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a55c15e5-3ebf-40cf-9345-3baf810d4a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:55.041208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:55.129675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940e5fa-1982-4091-b502-616388da2762 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:55.139329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49134f-8d92-4c23-8d3c-6f0997e85c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:55.140271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49134f-8d92-4c23-8d3c-6f0997e85c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:55.141827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49134f-8d92-4c23-8d3c-6f0997e85c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:55.142121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49134f-8d92-4c23-8d3c-6f0997e85c2c admin admin] PATH_INFO: `/v3/projects/f12f6358915b44ce8f2891e509214d50/users/f9cb2b5c72f74db9aac85848eb5dcf13/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.145759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49134f-8d92-4c23-8d3c-6f0997e85c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12f6358915b44ce8f2891e509214d50, user_id=f9cb2b5c72f74db9aac85848eb5dcf13, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:55.154885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49134f-8d92-4c23-8d3c-6f0997e85c2c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:55.158529 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 815/1633] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:01:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:55.159859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c15e5-3ebf-40cf-9345-3baf810d4a5d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:55.182899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 819/1634] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:54 2022] PUT /identity/v3/projects/f12f6358915b44ce8f2891e509214d50/users/f9cb2b5c72f74db9aac85848eb5dcf13/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:55.183152 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c15e5-3ebf-40cf-9345-3baf810d4a5d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:55.184916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49134f-8d92-4c23-8d3c-6f0997e85c2c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:55.289894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37c260-a9c9-4347-8bb8-665218d5672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:55.290792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37c260-a9c9-4347-8bb8-665218d5672b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:55.291139 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37c260-a9c9-4347-8bb8-665218d5672b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:55.291585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37c260-a9c9-4347-8bb8-665218d5672b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.292704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37c260-a9c9-4347-8bb8-665218d5672b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:55.292847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49134f-8d92-4c23-8d3c-6f0997e85c2c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:55.296014 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37c260-a9c9-4347-8bb8-665218d5672b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:55.298834 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07651fbb-f982-4d7f-a023-1eab620f5f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:55.299902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07651fbb-f982-4d7f-a023-1eab620f5f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:55.300293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07651fbb-f982-4d7f-a023-1eab620f5f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:55.300690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07651fbb-f982-4d7f-a023-1eab620f5f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.301573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07651fbb-f982-4d7f-a023-1eab620f5f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:55.303472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07651fbb-f982-4d7f-a023-1eab620f5f7b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:55.311780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 816/1635] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:55.313754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37c260-a9c9-4347-8bb8-665218d5672b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:55.316780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 820/1636] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:55.318779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae170b-cd7d-4260-9f30-4957306f36dd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:55.318978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae170b-cd7d-4260-9f30-4957306f36dd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:55.319238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae170b-cd7d-4260-9f30-4957306f36dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.357823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-acae170b-cd7d-4260-9f30-4957306f36dd None None] MFA Rules not processed for user `f9cb2b5c72f74db9aac85848eb5dcf13`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:55.359498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-acae170b-cd7d-4260-9f30-4957306f36dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:55.422327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37c260-a9c9-4347-8bb8-665218d5672b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:55.427423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7802d-e7b4-4b9c-8f15-f0c95e589606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:55.428238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7802d-e7b4-4b9c-8f15-f0c95e589606 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:55.428727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7802d-e7b4-4b9c-8f15-f0c95e589606 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:55.429634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7802d-e7b4-4b9c-8f15-f0c95e589606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.478426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:55.498496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 821/1637] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:55.498496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae170b-cd7d-4260-9f30-4957306f36dd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:55.517422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae170b-cd7d-4260-9f30-4957306f36dd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:55.525754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5336110-12b9-48aa-b3cf-50dbc06bd9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:55.527584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5336110-12b9-48aa-b3cf-50dbc06bd9fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:55.527584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5336110-12b9-48aa-b3cf-50dbc06bd9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:55.527847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5336110-12b9-48aa-b3cf-50dbc06bd9fe admin admin] PATH_INFO: `/v3/projects/a0b78286f8034622a190ce0bfc61b36c/users/f273ec6f8e544989b5a0383fd0ea383a/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.546974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5336110-12b9-48aa-b3cf-50dbc06bd9fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b78286f8034622a190ce0bfc61b36c, user_id=f273ec6f8e544989b5a0383fd0ea383a, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:55.551376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5336110-12b9-48aa-b3cf-50dbc06bd9fe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:55.584642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7802d-e7b4-4b9c-8f15-f0c95e589606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:55.591132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7802d-e7b4-4b9c-8f15-f0c95e589606 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:55.591588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 822/1638] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:55 2022] PUT /identity/v3/projects/a0b78286f8034622a190ce0bfc61b36c/users/f273ec6f8e544989b5a0383fd0ea383a/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:55.593323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5336110-12b9-48aa-b3cf-50dbc06bd9fe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:55.610849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 817/1639] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:55.612501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7802d-e7b4-4b9c-8f15-f0c95e589606 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:55.621427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7802d-e7b4-4b9c-8f15-f0c95e589606 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:55.657820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:55.728008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7556b8-1ce2-44ad-8989-938713c29419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:55.728684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7556b8-1ce2-44ad-8989-938713c29419 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:55.728973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7556b8-1ce2-44ad-8989-938713c29419 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:55.729258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7556b8-1ce2-44ad-8989-938713c29419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.755934 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5336110-12b9-48aa-b3cf-50dbc06bd9fe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:55.764023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e3675-cd6f-4d62-b69a-40b1b134475f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:55.764023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e3675-cd6f-4d62-b69a-40b1b134475f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:55.764023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e3675-cd6f-4d62-b69a-40b1b134475f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:55.764023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e3675-cd6f-4d62-b69a-40b1b134475f admin admin] PATH_INFO: `/v3/projects/0e8b6068752a4bee8bd24f49cd9094ca/users/944061fbbf244a688e427239f2710d10/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.765703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e3675-cd6f-4d62-b69a-40b1b134475f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8b6068752a4bee8bd24f49cd9094ca, user_id=944061fbbf244a688e427239f2710d10, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:55.769526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e3675-cd6f-4d62-b69a-40b1b134475f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:55.785025 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 823/1640] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:55 2022] PUT /identity/v3/projects/0e8b6068752a4bee8bd24f49cd9094ca/users/944061fbbf244a688e427239f2710d10/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:55.785941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e3675-cd6f-4d62-b69a-40b1b134475f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:55.891230 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e3675-cd6f-4d62-b69a-40b1b134475f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:55.896372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7556b8-1ce2-44ad-8989-938713c29419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:55.897045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ae8b7-c6c7-4c0c-b551-1b564a45dce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:55.897885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ae8b7-c6c7-4c0c-b551-1b564a45dce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:55.898175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ae8b7-c6c7-4c0c-b551-1b564a45dce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:55.898460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ae8b7-c6c7-4c0c-b551-1b564a45dce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.899208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ae8b7-c6c7-4c0c-b551-1b564a45dce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:55.899813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7556b8-1ce2-44ad-8989-938713c29419 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:55.900864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ae8b7-c6c7-4c0c-b551-1b564a45dce7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:55.913172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 824/1641] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:01:55.915332 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e09209-d756-4936-a9d5-79c131c4347c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:55.915970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e09209-d756-4936-a9d5-79c131c4347c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:55.916460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e09209-d756-4936-a9d5-79c131c4347c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.923297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 818/1642] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:55.924822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04145224-54dd-4583-9450-13bcac1bf564 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:55.925126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04145224-54dd-4583-9450-13bcac1bf564 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:55.925459 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04145224-54dd-4583-9450-13bcac1bf564 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.934734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 819/1643] 10.222.0.65 () {54 vars in 912 bytes} [Thu Jun 23 11:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:01:55.936957 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04145224-54dd-4583-9450-13bcac1bf564 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:55.963905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04145224-54dd-4583-9450-13bcac1bf564 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:55.973264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604fbc8-55ee-4336-bf4a-c1459e70623b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:55.974699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604fbc8-55ee-4336-bf4a-c1459e70623b admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:55.975062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604fbc8-55ee-4336-bf4a-c1459e70623b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:55.975566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604fbc8-55ee-4336-bf4a-c1459e70623b admin admin] PATH_INFO: `/v3/projects/a0b78286f8034622a190ce0bfc61b36c/users/f273ec6f8e544989b5a0383fd0ea383a/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:55.976901 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b7e09209-d756-4936-a9d5-79c131c4347c None None] MFA Rules not processed for user `944061fbbf244a688e427239f2710d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:55.979630 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604fbc8-55ee-4336-bf4a-c1459e70623b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b78286f8034622a190ce0bfc61b36c, user_id=f273ec6f8e544989b5a0383fd0ea383a, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:55.980786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b7e09209-d756-4936-a9d5-79c131c4347c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:55.983675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604fbc8-55ee-4336-bf4a-c1459e70623b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:56.016731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 820/1644] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:01:55 2022] PUT /identity/v3/projects/a0b78286f8034622a190ce0bfc61b36c/users/f273ec6f8e544989b5a0383fd0ea383a/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:56.021015 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d93c5-3984-45a6-beb8-0bd713819598 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:56.021349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d93c5-3984-45a6-beb8-0bd713819598 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:56.021650 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d93c5-3984-45a6-beb8-0bd713819598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:56.050041 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1b9d93c5-3984-45a6-beb8-0bd713819598 None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:56.051288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1b9d93c5-3984-45a6-beb8-0bd713819598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:56.209827 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:56.214600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 825/1645] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:55 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:56.216479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7664bd7-b3e1-4688-b915-45e936a90ad2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:56.216643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7664bd7-b3e1-4688-b915-45e936a90ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:56.217014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7664bd7-b3e1-4688-b915-45e936a90ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:56.261136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e7664bd7-b3e1-4688-b915-45e936a90ad2 None None] MFA Rules not processed for user `f273ec6f8e544989b5a0383fd0ea383a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:56.262434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e7664bd7-b3e1-4688-b915-45e936a90ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:56.265327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 821/1646] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:01:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:56.267991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d93c5-3984-45a6-beb8-0bd713819598 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:56.317181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:56.391469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 826/1647] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:56.392651 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7664bd7-b3e1-4688-b915-45e936a90ad2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:56.413114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d93c5-3984-45a6-beb8-0bd713819598 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:56.426582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed323e8e-f0f9-4455-8db1-69c098a00fa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:56.427815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed323e8e-f0f9-4455-8db1-69c098a00fa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:56.428115 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed323e8e-f0f9-4455-8db1-69c098a00fa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:56.428403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed323e8e-f0f9-4455-8db1-69c098a00fa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:56.469863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7664bd7-b3e1-4688-b915-45e936a90ad2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:56.481448 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81075b31-470d-42f5-9d9a-31d7e3d6df97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:56.483552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81075b31-470d-42f5-9d9a-31d7e3d6df97 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:56.483993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81075b31-470d-42f5-9d9a-31d7e3d6df97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:56.484397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81075b31-470d-42f5-9d9a-31d7e3d6df97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:56.555859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed323e8e-f0f9-4455-8db1-69c098a00fa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:56.558907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed323e8e-f0f9-4455-8db1-69c098a00fa7 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:56.571389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 822/1648] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:56.586328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81075b31-470d-42f5-9d9a-31d7e3d6df97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:56.590740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81075b31-470d-42f5-9d9a-31d7e3d6df97 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:56.604663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 827/1649] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:56 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:56.705327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed323e8e-f0f9-4455-8db1-69c098a00fa7 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:56.722371 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed323e8e-f0f9-4455-8db1-69c098a00fa7 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:56.730369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528b29c-391d-40ac-a9f3-8e500176bcdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:56.731186 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528b29c-391d-40ac-a9f3-8e500176bcdb service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:56.731648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528b29c-391d-40ac-a9f3-8e500176bcdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:56.731952 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528b29c-391d-40ac-a9f3-8e500176bcdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:56.754423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528b29c-391d-40ac-a9f3-8e500176bcdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:56.756266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528b29c-391d-40ac-a9f3-8e500176bcdb service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:56.790507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 823/1650] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:01:56.821669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81075b31-470d-42f5-9d9a-31d7e3d6df97 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:56.837360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81075b31-470d-42f5-9d9a-31d7e3d6df97 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:56.853093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16c067-bdec-4b80-957a-99d8a92aa9c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:56.854386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16c067-bdec-4b80-957a-99d8a92aa9c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:56.854851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16c067-bdec-4b80-957a-99d8a92aa9c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:56.855416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16c067-bdec-4b80-957a-99d8a92aa9c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:56.872842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16c067-bdec-4b80-957a-99d8a92aa9c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:56.874586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16c067-bdec-4b80-957a-99d8a92aa9c7 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:56.905338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 828/1651] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:57.152221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528b29c-391d-40ac-a9f3-8e500176bcdb service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:57.336304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528b29c-391d-40ac-a9f3-8e500176bcdb service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:57.349559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:57.350671 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:57.352316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:57.352556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] PATH_INFO: `/v3/users/800e9640bdab42bb9a8ea58eecc71653` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:57.356125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=800e9640bdab42bb9a8ea58eecc71653)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:57.360418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:57.434184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:57.482417 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '800e9640bdab42bb9a8ea58eecc71653'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:57.483151 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '800e9640bdab42bb9a8ea58eecc71653'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:57.500236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '800e9640bdab42bb9a8ea58eecc71653'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:57.528257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '800e9640bdab42bb9a8ea58eecc71653'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:57.552202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '800e9640bdab42bb9a8ea58eecc71653'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:57.566020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 824/1652] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:01:57 2022] DELETE /identity/v3/users/800e9640bdab42bb9a8ea58eecc71653 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:57.570873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43feb0-a7fc-48b3-9918-8a327fd611c8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:57.571700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43feb0-a7fc-48b3-9918-8a327fd611c8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:57.572057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43feb0-a7fc-48b3-9918-8a327fd611c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:57.611129 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ea43feb0-a7fc-48b3-9918-8a327fd611c8 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:57.612947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ea43feb0-a7fc-48b3-9918-8a327fd611c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:57.722866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:57.741659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 829/1653] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:57.801552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:57.803467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:57.819093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43feb0-a7fc-48b3-9918-8a327fd611c8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:57.820722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ea43feb0-a7fc-48b3-9918-8a327fd611c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:57.858767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43feb0-a7fc-48b3-9918-8a327fd611c8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:57.880113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadbd5b-b4d6-4424-85e0-21b7d5bb0acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:57.881537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadbd5b-b4d6-4424-85e0-21b7d5bb0acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:57.881674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadbd5b-b4d6-4424-85e0-21b7d5bb0acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:57.881941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadbd5b-b4d6-4424-85e0-21b7d5bb0acf admin admin] PATH_INFO: `/v3/users/ac6a4ac364db438494a8374781ce4ac3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:57.886975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eadbd5b-b4d6-4424-85e0-21b7d5bb0acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac6a4ac364db438494a8374781ce4ac3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:57.896727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-9eadbd5b-b4d6-4424-85e0-21b7d5bb0acf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:01:57.896727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 23 11:01:57.896727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: Jun 23 11:01:57.903856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eadbd5b-b4d6-4424-85e0-21b7d5bb0acf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:58.009557 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdc59e-4952-465d-990f-65a9f6842e19 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:58.021187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e32278-d09c-482e-a005-5594ccad3eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:58.021940 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e32278-d09c-482e-a005-5594ccad3eca service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:58.022200 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e32278-d09c-482e-a005-5594ccad3eca service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:58.022481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e32278-d09c-482e-a005-5594ccad3eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:58.056171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e32278-d09c-482e-a005-5594ccad3eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:58.057647 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e32278-d09c-482e-a005-5594ccad3eca service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:58.076233 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9eadbd5b-b4d6-4424-85e0-21b7d5bb0acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac6a4ac364db438494a8374781ce4ac3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:58.077200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9eadbd5b-b4d6-4424-85e0-21b7d5bb0acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac6a4ac364db438494a8374781ce4ac3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:58.079231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9eadbd5b-b4d6-4424-85e0-21b7d5bb0acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac6a4ac364db438494a8374781ce4ac3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:58.091611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 825/1654] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:58.096157 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e32278-d09c-482e-a005-5594ccad3eca service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:58.100483 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c4e32278-d09c-482e-a005-5594ccad3eca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:58.126968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9eadbd5b-b4d6-4424-85e0-21b7d5bb0acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac6a4ac364db438494a8374781ce4ac3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:58.134760 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e32278-d09c-482e-a005-5594ccad3eca service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:58.145894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4704141-da01-4fe8-aaa8-b6dd5b16ac76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:58.147246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4704141-da01-4fe8-aaa8-b6dd5b16ac76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:58.147246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4704141-da01-4fe8-aaa8-b6dd5b16ac76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:58.151926 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4704141-da01-4fe8-aaa8-b6dd5b16ac76 admin admin] PATH_INFO: `/v3/users/6d1188e9e13642449f694f50528f2723` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:58.154034 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9eadbd5b-b4d6-4424-85e0-21b7d5bb0acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac6a4ac364db438494a8374781ce4ac3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:58.154947 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4704141-da01-4fe8-aaa8-b6dd5b16ac76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d1188e9e13642449f694f50528f2723)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:58.159061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4704141-da01-4fe8-aaa8-b6dd5b16ac76 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:58.189767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 830/1655] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:01:57 2022] DELETE /identity/v3/users/ac6a4ac364db438494a8374781ce4ac3 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:01:58.203349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288204f3-2970-40ed-9b7f-2873647de7f1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:58.203554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288204f3-2970-40ed-9b7f-2873647de7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:58.203822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288204f3-2970-40ed-9b7f-2873647de7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:58.251196 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-288204f3-2970-40ed-9b7f-2873647de7f1 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:58.256453 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-288204f3-2970-40ed-9b7f-2873647de7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:58.277191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e4704141-da01-4fe8-aaa8-b6dd5b16ac76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d1188e9e13642449f694f50528f2723'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:58.278941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e4704141-da01-4fe8-aaa8-b6dd5b16ac76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d1188e9e13642449f694f50528f2723'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:58.296436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e4704141-da01-4fe8-aaa8-b6dd5b16ac76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d1188e9e13642449f694f50528f2723'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:58.367931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e4704141-da01-4fe8-aaa8-b6dd5b16ac76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d1188e9e13642449f694f50528f2723'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:58.388490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e4704141-da01-4fe8-aaa8-b6dd5b16ac76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d1188e9e13642449f694f50528f2723'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:01:58.406985 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 826/1656] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:01:58 2022] DELETE /identity/v3/users/6d1188e9e13642449f694f50528f2723 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:01:58.413372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d28c56-7084-4da6-a422-cb82c3c920f8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:58.413662 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d28c56-7084-4da6-a422-cb82c3c920f8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:58.414210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d28c56-7084-4da6-a422-cb82c3c920f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:58.425979 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 831/1657] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:58.458465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288204f3-2970-40ed-9b7f-2873647de7f1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:58.460051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-288204f3-2970-40ed-9b7f-2873647de7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:58.474097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-24d28c56-7084-4da6-a422-cb82c3c920f8 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:58.476576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-24d28c56-7084-4da6-a422-cb82c3c920f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:58.505147 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:58.726436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288204f3-2970-40ed-9b7f-2873647de7f1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:58.739422 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 827/1658] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:58.760809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d28c56-7084-4da6-a422-cb82c3c920f8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:58.762852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-24d28c56-7084-4da6-a422-cb82c3c920f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:58.804915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:58.875707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c09be0-6f59-48de-a252-f20260de0ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:58.877361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c09be0-6f59-48de-a252-f20260de0ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:58.877804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c09be0-6f59-48de-a252-f20260de0ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:58.878334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c09be0-6f59-48de-a252-f20260de0ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:58.925310 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c09be0-6f59-48de-a252-f20260de0ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:58.927358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c09be0-6f59-48de-a252-f20260de0ba8 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:58.944775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 832/1659] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:01:58.961165 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d28c56-7084-4da6-a422-cb82c3c920f8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:58.969536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e30bc-8a53-4f6e-8be1-153c5c95b619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:58.970551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e30bc-8a53-4f6e-8be1-153c5c95b619 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:58.970939 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e30bc-8a53-4f6e-8be1-153c5c95b619 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:58.971319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e30bc-8a53-4f6e-8be1-153c5c95b619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:59.004130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e30bc-8a53-4f6e-8be1-153c5c95b619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:59.008673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e30bc-8a53-4f6e-8be1-153c5c95b619 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:59.030202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 828/1660] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:00.136069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c09be0-6f59-48de-a252-f20260de0ba8 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:00.138828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a0c09be0-6f59-48de-a252-f20260de0ba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:00.175859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c09be0-6f59-48de-a252-f20260de0ba8 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:00.200472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da59925-7a2a-4b98-be90-4fb691d7c1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:00.203587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da59925-7a2a-4b98-be90-4fb691d7c1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:00.204115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da59925-7a2a-4b98-be90-4fb691d7c1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:00.204613 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da59925-7a2a-4b98-be90-4fb691d7c1e4 admin admin] PATH_INFO: `/v3/projects/82a06041149a40a6807070f31cf7225e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:00.207037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da59925-7a2a-4b98-be90-4fb691d7c1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82a06041149a40a6807070f31cf7225e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:00.213874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da59925-7a2a-4b98-be90-4fb691d7c1e4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:00.355239 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e30bc-8a53-4f6e-8be1-153c5c95b619 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:00.356877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-819e30bc-8a53-4f6e-8be1-153c5c95b619 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:00.361358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5da59925-7a2a-4b98-be90-4fb691d7c1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82a06041149a40a6807070f31cf7225e', 'request_id': 'req-5da59925-7a2a-4b98-be90-4fb691d7c1e4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:00.380291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5da59925-7a2a-4b98-be90-4fb691d7c1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82a06041149a40a6807070f31cf7225e', 'request_id': 'req-5da59925-7a2a-4b98-be90-4fb691d7c1e4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:00.383224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e30bc-8a53-4f6e-8be1-153c5c95b619 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:00.408309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5da59925-7a2a-4b98-be90-4fb691d7c1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82a06041149a40a6807070f31cf7225e', 'request_id': 'req-5da59925-7a2a-4b98-be90-4fb691d7c1e4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:00.448974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5da59925-7a2a-4b98-be90-4fb691d7c1e4 admin admin] The token cache is being invalidate because project 82a06041149a40a6807070f31cf7225e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:00.449285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5da59925-7a2a-4b98-be90-4fb691d7c1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:00.449992 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5da59925-7a2a-4b98-be90-4fb691d7c1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:00.452024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 833/1661] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:00 2022] DELETE /identity/v3/projects/82a06041149a40a6807070f31cf7225e => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:00.457863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31381de-11ec-45f1-a945-ef0fdaea6130 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:00.458159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31381de-11ec-45f1-a945-ef0fdaea6130 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:00.458493 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31381de-11ec-45f1-a945-ef0fdaea6130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:00.498551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b31381de-11ec-45f1-a945-ef0fdaea6130 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:00.500912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b31381de-11ec-45f1-a945-ef0fdaea6130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:00.629976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:00.634541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:00.637308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:00.638885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 834/1662] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:00.639448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] PATH_INFO: `/v3/projects/b7156540914b4b949c71dfc09f71e6ec` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:00.646855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32605a32-30cd-40a4-97e4-0813264c2738 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:00.647155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32605a32-30cd-40a4-97e4-0813264c2738 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:00.647426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32605a32-30cd-40a4-97e4-0813264c2738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:00.652322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7156540914b4b949c71dfc09f71e6ec)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:00.656171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:02:00.656171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 23 11:02:00.656171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: Jun 23 11:02:00.659967 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:00.696743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-32605a32-30cd-40a4-97e4-0813264c2738 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:00.704040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-32605a32-30cd-40a4-97e4-0813264c2738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:00.779856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 835/1663] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:00.787094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32605a32-30cd-40a4-97e4-0813264c2738 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:00.821932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32605a32-30cd-40a4-97e4-0813264c2738 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:00.840011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7156540914b4b949c71dfc09f71e6ec', 'request_id': 'req-d8d1722c-5701-4037-847d-4523e8870205'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:00.861499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7156540914b4b949c71dfc09f71e6ec', 'request_id': 'req-d8d1722c-5701-4037-847d-4523e8870205'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:00.869907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7156540914b4b949c71dfc09f71e6ec', 'request_id': 'req-d8d1722c-5701-4037-847d-4523e8870205'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:00.871848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:00.936236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] The token cache is being invalidate because project b7156540914b4b949c71dfc09f71e6ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:00.936534 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:00.951045 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:00.957820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 829/1664] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:00 2022] DELETE /identity/v3/projects/b7156540914b4b949c71dfc09f71e6ec => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:00.957820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:00.959979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:01.074219 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:01.143429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1722c-5701-4037-847d-4523e8870205 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:01.151138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:01.152214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:01.152510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:01.152945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] PATH_INFO: `/v3/projects/06e49e2ab21d4f82b2b10dd49330a703` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:01.154205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4713bd8-9e66-4bc5-bcdb-aa377ebd06f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:01.154642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e49e2ab21d4f82b2b10dd49330a703)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:01.154892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4713bd8-9e66-4bc5-bcdb-aa377ebd06f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:01.156931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:01.162640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4713bd8-9e66-4bc5-bcdb-aa377ebd06f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:01.163103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4713bd8-9e66-4bc5-bcdb-aa377ebd06f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:01.164411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4713bd8-9e66-4bc5-bcdb-aa377ebd06f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:01.167879 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4713bd8-9e66-4bc5-bcdb-aa377ebd06f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:01.173133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4713bd8-9e66-4bc5-bcdb-aa377ebd06f7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:01.224911 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 836/1665] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:01.226224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4713bd8-9e66-4bc5-bcdb-aa377ebd06f7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:01.228824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c4713bd8-9e66-4bc5-bcdb-aa377ebd06f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:01.260547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e49e2ab21d4f82b2b10dd49330a703', 'request_id': 'req-b02bb71e-5e7c-4a74-966e-b580b197605d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:01.276909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e49e2ab21d4f82b2b10dd49330a703', 'request_id': 'req-b02bb71e-5e7c-4a74-966e-b580b197605d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:01.291043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4713bd8-9e66-4bc5-bcdb-aa377ebd06f7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:01.296828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e49e2ab21d4f82b2b10dd49330a703', 'request_id': 'req-b02bb71e-5e7c-4a74-966e-b580b197605d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:01.332016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] The token cache is being invalidate because project 06e49e2ab21d4f82b2b10dd49330a703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:01.336029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:01.338801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:01.345370 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 830/1666] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:00 2022] DELETE /identity/v3/projects/06e49e2ab21d4f82b2b10dd49330a703 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:01.346707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:01.354684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:01.500471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:01.501090 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:01.501520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:01.501809 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] PATH_INFO: `/v3/users/944061fbbf244a688e427239f2710d10` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:01.504452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=944061fbbf244a688e427239f2710d10)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:01.506455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:01.582478 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bb71e-5e7c-4a74-966e-b580b197605d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:01.608980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6ceab-b9b9-46a9-8147-f7f28e4947fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:01.612323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6ceab-b9b9-46a9-8147-f7f28e4947fd admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:01.613346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6ceab-b9b9-46a9-8147-f7f28e4947fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:01.614523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6ceab-b9b9-46a9-8147-f7f28e4947fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:01.617394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6ceab-b9b9-46a9-8147-f7f28e4947fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:01.626737 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6ceab-b9b9-46a9-8147-f7f28e4947fd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:01.644329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944061fbbf244a688e427239f2710d10'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:01.645388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944061fbbf244a688e427239f2710d10'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:01.646193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944061fbbf244a688e427239f2710d10'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:01.667268 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944061fbbf244a688e427239f2710d10'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:01.701575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944061fbbf244a688e427239f2710d10'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:01.723545 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 831/1667] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:01 2022] POST /identity/v3/projects => generated 403 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:01.724616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6ceab-b9b9-46a9-8147-f7f28e4947fd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:01.726224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 837/1668] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:01 2022] DELETE /identity/v3/users/944061fbbf244a688e427239f2710d10 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:01.727098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:01.728594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:01.733735 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3cf6ceab-b9b9-46a9-8147-f7f28e4947fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:01.771710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:01.885534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9903fda-1f09-4007-b58c-d42421974c48 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:01.891277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e76009-ceb2-4055-a805-f7cf912724b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:01.892023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e76009-ceb2-4055-a805-f7cf912724b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:01.892312 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e76009-ceb2-4055-a805-f7cf912724b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:01.892597 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e76009-ceb2-4055-a805-f7cf912724b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:01.893490 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e76009-ceb2-4055-a805-f7cf912724b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:01.895502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e76009-ceb2-4055-a805-f7cf912724b9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:01.910445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:01.937165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 838/1669] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:01 2022] POST /identity/v3/users => generated 302 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:01.942312 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e76009-ceb2-4055-a805-f7cf912724b9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:01.965063 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e76009-ceb2-4055-a805-f7cf912724b9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:01.973209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6ceab-b9b9-46a9-8147-f7f28e4947fd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:01.985127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdba81-9213-477c-a404-06c64ab05ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:01.986126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdba81-9213-477c-a404-06c64ab05ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:01.987063 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdba81-9213-477c-a404-06c64ab05ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:01.987921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdba81-9213-477c-a404-06c64ab05ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:01.992322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c433e75-a1ad-4b5e-8534-3508110cdbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:01.993252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c433e75-a1ad-4b5e-8534-3508110cdbbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:01.993521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c433e75-a1ad-4b5e-8534-3508110cdbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:01.993685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdba81-9213-477c-a404-06c64ab05ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:01.993783 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c433e75-a1ad-4b5e-8534-3508110cdbbc admin admin] PATH_INFO: `/v3/users/f273ec6f8e544989b5a0383fd0ea383a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:01.997568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdba81-9213-477c-a404-06c64ab05ed8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:01.999978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c433e75-a1ad-4b5e-8534-3508110cdbbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f273ec6f8e544989b5a0383fd0ea383a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:02.005624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c433e75-a1ad-4b5e-8534-3508110cdbbc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:02.031882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 839/1670] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:02.035516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdba81-9213-477c-a404-06c64ab05ed8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:02.058479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdba81-9213-477c-a404-06c64ab05ed8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:02.072911 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3597b-1427-4e9d-a588-9f5be009b8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:02.073913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3597b-1427-4e9d-a588-9f5be009b8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:02.074235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3597b-1427-4e9d-a588-9f5be009b8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:02.074644 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3597b-1427-4e9d-a588-9f5be009b8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:02.075250 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3597b-1427-4e9d-a588-9f5be009b8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:02.079036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3597b-1427-4e9d-a588-9f5be009b8d5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:02.114332 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 840/1671] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:02.116411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3597b-1427-4e9d-a588-9f5be009b8d5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:02.118321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2c433e75-a1ad-4b5e-8534-3508110cdbbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f273ec6f8e544989b5a0383fd0ea383a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:02.119522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2c433e75-a1ad-4b5e-8534-3508110cdbbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f273ec6f8e544989b5a0383fd0ea383a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:02.138369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2c433e75-a1ad-4b5e-8534-3508110cdbbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f273ec6f8e544989b5a0383fd0ea383a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:02.141424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3597b-1427-4e9d-a588-9f5be009b8d5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:02.152175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ba02c-8914-403f-8e73-84be7ec40483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:02.153061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ba02c-8914-403f-8e73-84be7ec40483 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:02.153372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ba02c-8914-403f-8e73-84be7ec40483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:02.154475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ba02c-8914-403f-8e73-84be7ec40483 admin admin] PATH_INFO: `/v3/projects/d797911a43b4435bac05c3ec29c56d8d/users/b36e63ccb37244f7ad03cdee90591d7f/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:02.159598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2c433e75-a1ad-4b5e-8534-3508110cdbbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f273ec6f8e544989b5a0383fd0ea383a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:02.176576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2c433e75-a1ad-4b5e-8534-3508110cdbbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f273ec6f8e544989b5a0383fd0ea383a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:02.195411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 832/1672] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:01 2022] DELETE /identity/v3/users/f273ec6f8e544989b5a0383fd0ea383a => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:02.208436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ba02c-8914-403f-8e73-84be7ec40483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d797911a43b4435bac05c3ec29c56d8d, user_id=b36e63ccb37244f7ad03cdee90591d7f, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:02.212792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ba02c-8914-403f-8e73-84be7ec40483 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:02.272478 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9ba01-c5fa-485a-9b06-feff23aaff4c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:02.272790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9ba01-c5fa-485a-9b06-feff23aaff4c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:02.273071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9ba01-c5fa-485a-9b06-feff23aaff4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:02.273460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 841/1673] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:02 2022] PUT /identity/v3/projects/d797911a43b4435bac05c3ec29c56d8d/users/b36e63ccb37244f7ad03cdee90591d7f/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:02.329783 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-15e9ba01-c5fa-485a-9b06-feff23aaff4c None None] MFA Rules not processed for user `b36e63ccb37244f7ad03cdee90591d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:02.331048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-15e9ba01-c5fa-485a-9b06-feff23aaff4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:02.505460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 833/1674] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:02.510113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ba02c-8914-403f-8e73-84be7ec40483 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:02.512118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bc3ba02c-8914-403f-8e73-84be7ec40483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:02.716328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ba02c-8914-403f-8e73-84be7ec40483 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:02.728865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57259f48-705e-459a-9cb5-a4f4c8fc6338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:02.732512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57259f48-705e-459a-9cb5-a4f4c8fc6338 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:02.732951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57259f48-705e-459a-9cb5-a4f4c8fc6338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:02.733403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57259f48-705e-459a-9cb5-a4f4c8fc6338 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:02.734761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57259f48-705e-459a-9cb5-a4f4c8fc6338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:02.740287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57259f48-705e-459a-9cb5-a4f4c8fc6338 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:02.808499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 842/1675] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:02 2022] POST /identity/v3/projects => generated 405 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:02.809709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9ba01-c5fa-485a-9b06-feff23aaff4c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:02.837539 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9ba01-c5fa-485a-9b06-feff23aaff4c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:02.857259 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:02.922522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e1b57-999d-4701-a488-b6535c83978a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:02.923204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e1b57-999d-4701-a488-b6535c83978a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:02.923498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e1b57-999d-4701-a488-b6535c83978a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:02.923822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e1b57-999d-4701-a488-b6535c83978a admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:02.924663 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e1b57-999d-4701-a488-b6535c83978a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:02.927004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e1b57-999d-4701-a488-b6535c83978a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:02.974745 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 834/1676] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:02 2022] POST /identity/v3/users => generated 309 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:02.977569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57259f48-705e-459a-9cb5-a4f4c8fc6338 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:02.994628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57259f48-705e-459a-9cb5-a4f4c8fc6338 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:03.009905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cf482-7228-4962-97c4-089fd338dce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:03.011553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cf482-7228-4962-97c4-089fd338dce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:03.012027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cf482-7228-4962-97c4-089fd338dce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:03.012753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cf482-7228-4962-97c4-089fd338dce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:03.013634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cf482-7228-4962-97c4-089fd338dce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:03.018002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cf482-7228-4962-97c4-089fd338dce2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:03.048805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e1b57-999d-4701-a488-b6535c83978a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:03.051879 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 843/1677] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:03.065889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e1b57-999d-4701-a488-b6535c83978a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:03.077770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37578d-5073-40e1-8326-a7aae489e991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:03.080885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37578d-5073-40e1-8326-a7aae489e991 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:03.081184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37578d-5073-40e1-8326-a7aae489e991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:03.081466 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37578d-5073-40e1-8326-a7aae489e991 admin admin] PATH_INFO: `/v3/projects/b70212ee7b494d638957cd5771b14f18/users/6a6bbeba0fe247949ffa22d24e8ff429/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:03.147400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37578d-5073-40e1-8326-a7aae489e991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b70212ee7b494d638957cd5771b14f18, user_id=6a6bbeba0fe247949ffa22d24e8ff429, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:03.150919 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37578d-5073-40e1-8326-a7aae489e991 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:03.192551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 835/1678] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:03 2022] PUT /identity/v3/projects/b70212ee7b494d638957cd5771b14f18/users/6a6bbeba0fe247949ffa22d24e8ff429/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:03.194766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cf482-7228-4962-97c4-089fd338dce2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:03.208997 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cf482-7228-4962-97c4-089fd338dce2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:03.300216 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87dab3-4a28-4ee7-8f4b-4599f01a2be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:03.300843 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87dab3-4a28-4ee7-8f4b-4599f01a2be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:03.301123 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87dab3-4a28-4ee7-8f4b-4599f01a2be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:03.301404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87dab3-4a28-4ee7-8f4b-4599f01a2be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:03.302133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87dab3-4a28-4ee7-8f4b-4599f01a2be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:03.303770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87dab3-4a28-4ee7-8f4b-4599f01a2be0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:03.319054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 844/1679] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:03.320335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37578d-5073-40e1-8326-a7aae489e991 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:03.342132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37578d-5073-40e1-8326-a7aae489e991 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:03.358346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d38ce-6570-41bb-b444-d1ff81f0d482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:03.370947 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d38ce-6570-41bb-b444-d1ff81f0d482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:03.371973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d38ce-6570-41bb-b444-d1ff81f0d482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:03.372806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d38ce-6570-41bb-b444-d1ff81f0d482 admin admin] PATH_INFO: `/v3/projects/b70212ee7b494d638957cd5771b14f18/users/6a6bbeba0fe247949ffa22d24e8ff429/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:03.381353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d38ce-6570-41bb-b444-d1ff81f0d482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b70212ee7b494d638957cd5771b14f18, user_id=6a6bbeba0fe247949ffa22d24e8ff429, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:03.396095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d38ce-6570-41bb-b444-d1ff81f0d482 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:03.439486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 836/1680] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:03 2022] PUT /identity/v3/projects/b70212ee7b494d638957cd5771b14f18/users/6a6bbeba0fe247949ffa22d24e8ff429/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:03.453523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896486a-c7a4-4828-b8ff-e84feb2fc684 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:03.455542 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896486a-c7a4-4828-b8ff-e84feb2fc684 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:03.456093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896486a-c7a4-4828-b8ff-e84feb2fc684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:03.505029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:03.517311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6896486a-c7a4-4828-b8ff-e84feb2fc684 None None] MFA Rules not processed for user `6a6bbeba0fe247949ffa22d24e8ff429`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:03.518655 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6896486a-c7a4-4828-b8ff-e84feb2fc684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:03.541677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d38ce-6570-41bb-b444-d1ff81f0d482 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:03.546596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d13d38ce-6570-41bb-b444-d1ff81f0d482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:03.666266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 845/1681] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:03.670040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896486a-c7a4-4828-b8ff-e84feb2fc684 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:03.672563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6896486a-c7a4-4828-b8ff-e84feb2fc684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:03.690563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d38ce-6570-41bb-b444-d1ff81f0d482 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:03.700004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:03.701108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:03.701535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:03.701765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896486a-c7a4-4828-b8ff-e84feb2fc684 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:03.702129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] PATH_INFO: `/v3/projects/a0b78286f8034622a190ce0bfc61b36c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:03.703480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b78286f8034622a190ce0bfc61b36c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:03.705589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:03.707314 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da74f4-d59f-447e-a768-bef00cebbb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:03.708229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da74f4-d59f-447e-a768-bef00cebbb87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:03.708521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da74f4-d59f-447e-a768-bef00cebbb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:03.708783 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da74f4-d59f-447e-a768-bef00cebbb87 admin admin] PATH_INFO: `/v3/projects/0e8b6068752a4bee8bd24f49cd9094ca` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:03.712463 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da74f4-d59f-447e-a768-bef00cebbb87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e8b6068752a4bee8bd24f49cd9094ca)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:03.714911 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da74f4-d59f-447e-a768-bef00cebbb87 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:03.815249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b78286f8034622a190ce0bfc61b36c', 'request_id': 'req-f60d404b-8589-4b70-894d-a33f4ec0a805'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:03.826613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b78286f8034622a190ce0bfc61b36c', 'request_id': 'req-f60d404b-8589-4b70-894d-a33f4ec0a805'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:03.841960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b78286f8034622a190ce0bfc61b36c', 'request_id': 'req-f60d404b-8589-4b70-894d-a33f4ec0a805'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:03.851615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-28da74f4-d59f-447e-a768-bef00cebbb87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e8b6068752a4bee8bd24f49cd9094ca', 'request_id': 'req-28da74f4-d59f-447e-a768-bef00cebbb87'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:03.860888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:03.861974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-28da74f4-d59f-447e-a768-bef00cebbb87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e8b6068752a4bee8bd24f49cd9094ca', 'request_id': 'req-28da74f4-d59f-447e-a768-bef00cebbb87'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:03.868342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] The token cache is being invalidate because project a0b78286f8034622a190ce0bfc61b36c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:03.869419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:03.870690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:03.878512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 837/1682] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:03 2022] DELETE /identity/v3/projects/a0b78286f8034622a190ce0bfc61b36c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:03.880074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:03.881899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-28da74f4-d59f-447e-a768-bef00cebbb87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e8b6068752a4bee8bd24f49cd9094ca', 'request_id': 'req-28da74f4-d59f-447e-a768-bef00cebbb87'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:03.882869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:03.914339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-28da74f4-d59f-447e-a768-bef00cebbb87 admin admin] The token cache is being invalidate because project 0e8b6068752a4bee8bd24f49cd9094ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:03.914772 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-28da74f4-d59f-447e-a768-bef00cebbb87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:03.915697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-28da74f4-d59f-447e-a768-bef00cebbb87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:03.924416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 846/1683] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:03 2022] DELETE /identity/v3/projects/0e8b6068752a4bee8bd24f49cd9094ca => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:03.925990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5140e4-154b-4b5e-bc14-ee2236be8fb1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:03.927443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5140e4-154b-4b5e-bc14-ee2236be8fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:03.929778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5140e4-154b-4b5e-bc14-ee2236be8fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:03.967431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7d5140e4-154b-4b5e-bc14-ee2236be8fb1 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:03.969883 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7d5140e4-154b-4b5e-bc14-ee2236be8fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:04.040431 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d404b-8589-4b70-894d-a33f4ec0a805 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:04.127008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 847/1684] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:04.137160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06eab62-b635-468c-ba1c-b95fffd32e17 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.137880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06eab62-b635-468c-ba1c-b95fffd32e17 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.139973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06eab62-b635-468c-ba1c-b95fffd32e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.169225 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb40898-2635-40c1-b9e8-b2837195dfa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:04.169982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb40898-2635-40c1-b9e8-b2837195dfa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.170395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb40898-2635-40c1-b9e8-b2837195dfa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.170766 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb40898-2635-40c1-b9e8-b2837195dfa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.173135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d06eab62-b635-468c-ba1c-b95fffd32e17 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:04.175353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9cb40898-2635-40c1-b9e8-b2837195dfa7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:04.177156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d06eab62-b635-468c-ba1c-b95fffd32e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:04.203397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 848/1685] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:04.215431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d114bb-2d32-4e55-a61e-2b283abcc6f9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.215753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d114bb-2d32-4e55-a61e-2b283abcc6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.216023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d114bb-2d32-4e55-a61e-2b283abcc6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.257787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a0d114bb-2d32-4e55-a61e-2b283abcc6f9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:04.263779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a0d114bb-2d32-4e55-a61e-2b283abcc6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:04.286581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 849/1686] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:04.291795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3473ba-650b-460e-926c-069728feb15b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.292088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3473ba-650b-460e-926c-069728feb15b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.292378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3473ba-650b-460e-926c-069728feb15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.332664 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb40898-2635-40c1-b9e8-b2837195dfa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:04.334157 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb40898-2635-40c1-b9e8-b2837195dfa7 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:04.336740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1e3473ba-650b-460e-926c-069728feb15b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:04.337965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1e3473ba-650b-460e-926c-069728feb15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:04.374306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 838/1688] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:03 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:04.374566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 850/1688] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:04.376582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3473ba-650b-460e-926c-069728feb15b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:04.376988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb40898-2635-40c1-b9e8-b2837195dfa7 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:04.384150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:04.400678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3473ba-650b-460e-926c-069728feb15b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:04.402503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb40898-2635-40c1-b9e8-b2837195dfa7 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:04.411695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d330b-3e48-4fdf-a7ce-c1036ea2ae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:04.412374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d1f68-03c0-463b-9c94-330422eea6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:04.413041 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d1f68-03c0-463b-9c94-330422eea6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.413877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d1f68-03c0-463b-9c94-330422eea6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.414148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d1f68-03c0-463b-9c94-330422eea6a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.415085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d330b-3e48-4fdf-a7ce-c1036ea2ae39 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.415368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d330b-3e48-4fdf-a7ce-c1036ea2ae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.415640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d330b-3e48-4fdf-a7ce-c1036ea2ae39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.416397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d330b-3e48-4fdf-a7ce-c1036ea2ae39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:04.416760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d330b-3e48-4fdf-a7ce-c1036ea2ae39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:04.418364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d330b-3e48-4fdf-a7ce-c1036ea2ae39 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:04.420498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d1f68-03c0-463b-9c94-330422eea6a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:04.420818 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d1f68-03c0-463b-9c94-330422eea6a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:04.422442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d1f68-03c0-463b-9c94-330422eea6a7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:04.444944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 851/1689] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:04.456686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d330b-3e48-4fdf-a7ce-c1036ea2ae39 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:04.457779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 839/1690] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:04.461895 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d1f68-03c0-463b-9c94-330422eea6a7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:04.480571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d330b-3e48-4fdf-a7ce-c1036ea2ae39 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:04.483315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d1f68-03c0-463b-9c94-330422eea6a7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:04.491978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4823668-6388-4c35-93fa-d1486eda93b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:04.492674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4823668-6388-4c35-93fa-d1486eda93b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.492953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4823668-6388-4c35-93fa-d1486eda93b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.493235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4823668-6388-4c35-93fa-d1486eda93b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.493947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4823668-6388-4c35-93fa-d1486eda93b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:04.496952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4823668-6388-4c35-93fa-d1486eda93b0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:04.498813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071c842-b72d-41f5-a87b-3b577b1aa0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:04.499838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071c842-b72d-41f5-a87b-3b577b1aa0f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.500124 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071c842-b72d-41f5-a87b-3b577b1aa0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.500397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071c842-b72d-41f5-a87b-3b577b1aa0f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.501103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071c842-b72d-41f5-a87b-3b577b1aa0f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:04.503284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071c842-b72d-41f5-a87b-3b577b1aa0f7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:04.561157 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 840/1691] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:04 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:04.561320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 852/1692] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:04 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:04.563194 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4823668-6388-4c35-93fa-d1486eda93b0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:04.563866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071c842-b72d-41f5-a87b-3b577b1aa0f7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:04.609897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:04.653813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071c842-b72d-41f5-a87b-3b577b1aa0f7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:04.791348 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbeea3-98d6-470c-a8f2-96ff16606e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:04.793001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbeea3-98d6-470c-a8f2-96ff16606e26 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.793361 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbeea3-98d6-470c-a8f2-96ff16606e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.793711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbeea3-98d6-470c-a8f2-96ff16606e26 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.794491 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbeea3-98d6-470c-a8f2-96ff16606e26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:04.796563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbeea3-98d6-470c-a8f2-96ff16606e26 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:04.824758 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4823668-6388-4c35-93fa-d1486eda93b0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:04.832647 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 841/1693] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:04 2022] POST /identity/v3/users => generated 294 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:04.839564 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbeea3-98d6-470c-a8f2-96ff16606e26 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:04.847526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10c4db-833d-4bf7-91c9-7678a6c2fe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:04.848391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10c4db-833d-4bf7-91c9-7678a6c2fe02 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.849776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10c4db-833d-4bf7-91c9-7678a6c2fe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.854616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10c4db-833d-4bf7-91c9-7678a6c2fe02 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.862981 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10c4db-833d-4bf7-91c9-7678a6c2fe02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:04.865115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10c4db-833d-4bf7-91c9-7678a6c2fe02 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:04.875394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbeea3-98d6-470c-a8f2-96ff16606e26 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:04.902355 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e68a7e-68c5-407f-8ae3-9f9604d64f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:04.903577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e68a7e-68c5-407f-8ae3-9f9604d64f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.905044 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e68a7e-68c5-407f-8ae3-9f9604d64f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.905400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e68a7e-68c5-407f-8ae3-9f9604d64f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.908543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e68a7e-68c5-407f-8ae3-9f9604d64f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:04.912960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e68a7e-68c5-407f-8ae3-9f9604d64f33 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:04.916995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 853/1694] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:04 2022] POST /identity/v3/users => generated 284 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:04.921059 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10c4db-833d-4bf7-91c9-7678a6c2fe02 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:04.950557 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 842/1695] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:04.951594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e68a7e-68c5-407f-8ae3-9f9604d64f33 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:04.953303 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10c4db-833d-4bf7-91c9-7678a6c2fe02 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:04.968979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e68a7e-68c5-407f-8ae3-9f9604d64f33 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:04.974554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c4209-6d8d-434d-9691-3aaa33340aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:04.975742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898fba1-1bb9-4ddc-812f-78e7c4021b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:04.976475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898fba1-1bb9-4ddc-812f-78e7c4021b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.977538 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898fba1-1bb9-4ddc-812f-78e7c4021b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.977538 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898fba1-1bb9-4ddc-812f-78e7c4021b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.978005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c4209-6d8d-434d-9691-3aaa33340aaa admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:04.978478 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c4209-6d8d-434d-9691-3aaa33340aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:04.978932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c4209-6d8d-434d-9691-3aaa33340aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:04.979895 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c4209-6d8d-434d-9691-3aaa33340aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:04.980218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898fba1-1bb9-4ddc-812f-78e7c4021b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:04.981957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898fba1-1bb9-4ddc-812f-78e7c4021b63 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:04.982304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c4209-6d8d-434d-9691-3aaa33340aaa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:05.020041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 854/1696] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:05.024829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898fba1-1bb9-4ddc-812f-78e7c4021b63 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:05.030549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 843/1697] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:05.035323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c4209-6d8d-434d-9691-3aaa33340aaa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:05.045982 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898fba1-1bb9-4ddc-812f-78e7c4021b63 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:05.058067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c4209-6d8d-434d-9691-3aaa33340aaa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:05.062877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684c2c4-4bc8-49d2-ac6a-23d4806a3d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:05.063509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684c2c4-4bc8-49d2-ac6a-23d4806a3d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:05.063884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684c2c4-4bc8-49d2-ac6a-23d4806a3d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:05.064298 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684c2c4-4bc8-49d2-ac6a-23d4806a3d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:05.065580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baae756-ee41-45b8-9727-27e6e23cd0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:05.066248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684c2c4-4bc8-49d2-ac6a-23d4806a3d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:05.066508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baae756-ee41-45b8-9727-27e6e23cd0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:05.067306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baae756-ee41-45b8-9727-27e6e23cd0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:05.067306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baae756-ee41-45b8-9727-27e6e23cd0d2 admin admin] PATH_INFO: `/v3/projects/73388109969f4bb9b1ba7b5e61edb813/users/f5c75bc77eb94ca9bc4766aafb39eba9/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:05.070352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684c2c4-4bc8-49d2-ac6a-23d4806a3d4d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:05.102249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baae756-ee41-45b8-9727-27e6e23cd0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73388109969f4bb9b1ba7b5e61edb813, user_id=f5c75bc77eb94ca9bc4766aafb39eba9, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:05.105895 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baae756-ee41-45b8-9727-27e6e23cd0d2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:05.109643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 855/1698] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:05.114076 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684c2c4-4bc8-49d2-ac6a-23d4806a3d4d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:05.137771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 844/1699] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:05 2022] PUT /identity/v3/projects/73388109969f4bb9b1ba7b5e61edb813/users/f5c75bc77eb94ca9bc4766aafb39eba9/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:05.145677 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684c2c4-4bc8-49d2-ac6a-23d4806a3d4d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:05.149728 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1a71c-9dfa-4667-8fca-70853a8c4590 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:05.150645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1a71c-9dfa-4667-8fca-70853a8c4590 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:05.150961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1a71c-9dfa-4667-8fca-70853a8c4590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:05.225426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dcf1a71c-9dfa-4667-8fca-70853a8c4590 None None] MFA Rules not processed for user `f5c75bc77eb94ca9bc4766aafb39eba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:05.230375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dcf1a71c-9dfa-4667-8fca-70853a8c4590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:05.285397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b2fba-a785-4fae-a5e0-117c0d62cc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:05.286931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b2fba-a785-4fae-a5e0-117c0d62cc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:05.287412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b2fba-a785-4fae-a5e0-117c0d62cc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:05.287945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b2fba-a785-4fae-a5e0-117c0d62cc69 admin admin] PATH_INFO: `/v3/projects/87306ef18d364b5fabb1687ce5ca3280/users/da4f35033ad54f799c3727c7f83cffd1/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:05.331614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b2fba-a785-4fae-a5e0-117c0d62cc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87306ef18d364b5fabb1687ce5ca3280, user_id=da4f35033ad54f799c3727c7f83cffd1, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:05.334695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b2fba-a785-4fae-a5e0-117c0d62cc69 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:05.370694 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 856/1700] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:05 2022] PUT /identity/v3/projects/87306ef18d364b5fabb1687ce5ca3280/users/da4f35033ad54f799c3727c7f83cffd1/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:05.384101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef8e66-7d98-4b23-b996-9c2c0a7e13f1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:05.385309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef8e66-7d98-4b23-b996-9c2c0a7e13f1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:05.385791 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef8e66-7d98-4b23-b996-9c2c0a7e13f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:05.412823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 845/1701] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:05.414603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1a71c-9dfa-4667-8fca-70853a8c4590 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:05.418901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dcf1a71c-9dfa-4667-8fca-70853a8c4590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:05.452972 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1a71c-9dfa-4667-8fca-70853a8c4590 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:05.458095 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2bef8e66-7d98-4b23-b996-9c2c0a7e13f1 None None] MFA Rules not processed for user `da4f35033ad54f799c3727c7f83cffd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:05.462311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2bef8e66-7d98-4b23-b996-9c2c0a7e13f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:05.463355 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:05.474600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:05.610344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:05.612272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:05.612650 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:05.613156 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] PATH_INFO: `/v3/users/b36e63ccb37244f7ad03cdee90591d7f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:05.615087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b36e63ccb37244f7ad03cdee90591d7f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:05.617905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:05.656562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 857/1702] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:05.657970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef8e66-7d98-4b23-b996-9c2c0a7e13f1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:05.685552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef8e66-7d98-4b23-b996-9c2c0a7e13f1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:05.697014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7789b-a0c0-4039-9b25-0a2a8b55b22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:05.697497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b36e63ccb37244f7ad03cdee90591d7f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:05.697791 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7789b-a0c0-4039-9b25-0a2a8b55b22e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:05.698079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7789b-a0c0-4039-9b25-0a2a8b55b22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:05.698357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7789b-a0c0-4039-9b25-0a2a8b55b22e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:05.698472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b36e63ccb37244f7ad03cdee90591d7f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:05.699065 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7789b-a0c0-4039-9b25-0a2a8b55b22e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:05.701050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7789b-a0c0-4039-9b25-0a2a8b55b22e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:05.713617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b36e63ccb37244f7ad03cdee90591d7f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:05.731981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b36e63ccb37244f7ad03cdee90591d7f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:05.757264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b36e63ccb37244f7ad03cdee90591d7f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:05.767372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 858/1703] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:05 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:05.767372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7789b-a0c0-4039-9b25-0a2a8b55b22e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:05.767841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 846/1704] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:05 2022] DELETE /identity/v3/users/b36e63ccb37244f7ad03cdee90591d7f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:05.768814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:05.769772 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-daa7789b-a0c0-4039-9b25-0a2a8b55b22e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:05.770443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:05.890615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988b580-dce4-4dc3-b5f7-36ed239f66d2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:05.897468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07616529-cd1b-40d8-a17b-efb3329c606e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:05.898744 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07616529-cd1b-40d8-a17b-efb3329c606e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:05.899104 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07616529-cd1b-40d8-a17b-efb3329c606e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:05.899460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07616529-cd1b-40d8-a17b-efb3329c606e admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:05.900991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07616529-cd1b-40d8-a17b-efb3329c606e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:05.903268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07616529-cd1b-40d8-a17b-efb3329c606e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:05.942927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 847/1705] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:05 2022] POST /identity/v3/users => generated 300 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:05.944730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e6f4-a527-4931-bd94-12c55817b6ac None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:05.945033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e6f4-a527-4931-bd94-12c55817b6ac None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:05.945442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e6f4-a527-4931-bd94-12c55817b6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:05.974239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7789b-a0c0-4039-9b25-0a2a8b55b22e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:05.985830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba94e52-2893-4f18-abf3-cfa3a7dc69bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:05.987012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba94e52-2893-4f18-abf3-cfa3a7dc69bd admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:05.987307 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba94e52-2893-4f18-abf3-cfa3a7dc69bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:05.987588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba94e52-2893-4f18-abf3-cfa3a7dc69bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:05.990262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba94e52-2893-4f18-abf3-cfa3a7dc69bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:05.995153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba94e52-2893-4f18-abf3-cfa3a7dc69bd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:06.003349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d048e6f4-a527-4931-bd94-12c55817b6ac None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:06.005538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d048e6f4-a527-4931-bd94-12c55817b6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:06.033597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 848/1706] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:06.033597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048e6f4-a527-4931-bd94-12c55817b6ac None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:06.056394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 859/1707] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:05 2022] POST /identity/v3/projects => generated 369 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:06.060804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba94e52-2893-4f18-abf3-cfa3a7dc69bd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:06.068923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048e6f4-a527-4931-bd94-12c55817b6ac None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:06.106712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:06.216360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba94e52-2893-4f18-abf3-cfa3a7dc69bd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:06.238276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb0b96-48ab-4b85-a017-50536c806e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:06.239375 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb0b96-48ab-4b85-a017-50536c806e0b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.239834 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb0b96-48ab-4b85-a017-50536c806e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.240275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb0b96-48ab-4b85-a017-50536c806e0b admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.242720 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb0b96-48ab-4b85-a017-50536c806e0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:06.247491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb0b96-48ab-4b85-a017-50536c806e0b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:06.276060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcf55b-4662-4ed5-b10f-4c3171a34f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:06.276984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcf55b-4662-4ed5-b10f-4c3171a34f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.278276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcf55b-4662-4ed5-b10f-4c3171a34f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.279373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcf55b-4662-4ed5-b10f-4c3171a34f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.279915 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcf55b-4662-4ed5-b10f-4c3171a34f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:06.287110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcf55b-4662-4ed5-b10f-4c3171a34f50 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:06.293703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:06.301906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 860/1708] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:06 2022] POST /identity/v3/users => generated 291 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:06.309558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb0b96-48ab-4b85-a017-50536c806e0b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:06.314405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-35cb0b96-48ab-4b85-a017-50536c806e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:06.326005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 849/1709] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:06.332905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcf55b-4662-4ed5-b10f-4c3171a34f50 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:06.358395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcf55b-4662-4ed5-b10f-4c3171a34f50 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:06.367701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6f93a-7ae2-44fd-978a-74a8206d1292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:06.370168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6f93a-7ae2-44fd-978a-74a8206d1292 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.372128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6f93a-7ae2-44fd-978a-74a8206d1292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.372495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6f93a-7ae2-44fd-978a-74a8206d1292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.374234 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6f93a-7ae2-44fd-978a-74a8206d1292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:06.375541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6f93a-7ae2-44fd-978a-74a8206d1292 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:06.416064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 850/1710] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:06.417375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6f93a-7ae2-44fd-978a-74a8206d1292 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:06.440792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6f93a-7ae2-44fd-978a-74a8206d1292 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:06.453569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93834944-588a-4ad1-8819-eaffe0303285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:06.455337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93834944-588a-4ad1-8819-eaffe0303285 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.455632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93834944-588a-4ad1-8819-eaffe0303285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.455951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93834944-588a-4ad1-8819-eaffe0303285 admin admin] PATH_INFO: `/v3/projects/a1136b2daaba4d58aae0024c215d2843/users/26b01f17d3b4463fad83821410187164/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.481300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb0b96-48ab-4b85-a017-50536c806e0b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:06.486191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddf3a1-9a67-44bf-bc6b-790f1025d609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:06.486865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddf3a1-9a67-44bf-bc6b-790f1025d609 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.487156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddf3a1-9a67-44bf-bc6b-790f1025d609 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.487442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddf3a1-9a67-44bf-bc6b-790f1025d609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.495722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93834944-588a-4ad1-8819-eaffe0303285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1136b2daaba4d58aae0024c215d2843, user_id=26b01f17d3b4463fad83821410187164, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:06.502020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddf3a1-9a67-44bf-bc6b-790f1025d609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:06.502291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93834944-588a-4ad1-8819-eaffe0303285 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:06.505708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddf3a1-9a67-44bf-bc6b-790f1025d609 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:06.533809 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 861/1711] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:06.534988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddf3a1-9a67-44bf-bc6b-790f1025d609 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:06.539259 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 851/1712] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:06 2022] PUT /identity/v3/projects/a1136b2daaba4d58aae0024c215d2843/users/26b01f17d3b4463fad83821410187164/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:06.540224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93834944-588a-4ad1-8819-eaffe0303285 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:06.557492 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddf3a1-9a67-44bf-bc6b-790f1025d609 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:06.680217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93834944-588a-4ad1-8819-eaffe0303285 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:06.696694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03f886-780a-44bd-98ac-2107b324db3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:06.697454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03f886-780a-44bd-98ac-2107b324db3e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.698063 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03f886-780a-44bd-98ac-2107b324db3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.698357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03f886-780a-44bd-98ac-2107b324db3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.699262 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03f886-780a-44bd-98ac-2107b324db3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:06.700931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03f886-780a-44bd-98ac-2107b324db3e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:06.703036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106ef8c-ad56-4148-906b-e540157cf3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:06.704251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106ef8c-ad56-4148-906b-e540157cf3e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.704624 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106ef8c-ad56-4148-906b-e540157cf3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.705143 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106ef8c-ad56-4148-906b-e540157cf3e1 admin admin] PATH_INFO: `/v3/projects/8057f6b1fb7b4efa883c29501728893e/users/42e59bf8ecc1402381d2260f518a6aa5/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.725849 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 852/1713] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:06.726935 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03f886-780a-44bd-98ac-2107b324db3e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:06.730392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106ef8c-ad56-4148-906b-e540157cf3e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8057f6b1fb7b4efa883c29501728893e, user_id=42e59bf8ecc1402381d2260f518a6aa5, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:06.736987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106ef8c-ad56-4148-906b-e540157cf3e1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:06.737577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03f886-780a-44bd-98ac-2107b324db3e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:06.748575 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0df54a-823b-40df-aec6-e078034e5294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:06.749494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0df54a-823b-40df-aec6-e078034e5294 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.749780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0df54a-823b-40df-aec6-e078034e5294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.750060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0df54a-823b-40df-aec6-e078034e5294 admin admin] PATH_INFO: `/v3/projects/a1136b2daaba4d58aae0024c215d2843/users/26b01f17d3b4463fad83821410187164/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.753430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0df54a-823b-40df-aec6-e078034e5294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1136b2daaba4d58aae0024c215d2843, user_id=26b01f17d3b4463fad83821410187164, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:06.760458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0df54a-823b-40df-aec6-e078034e5294 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:06.765999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 862/1714] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:06 2022] PUT /identity/v3/projects/8057f6b1fb7b4efa883c29501728893e/users/42e59bf8ecc1402381d2260f518a6aa5/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:06.769706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106ef8c-ad56-4148-906b-e540157cf3e1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:06.793478 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 853/1715] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:06 2022] PUT /identity/v3/projects/a1136b2daaba4d58aae0024c215d2843/users/26b01f17d3b4463fad83821410187164/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:06.805387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8944f3-2d7f-486e-8e85-841729f326a5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.805678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8944f3-2d7f-486e-8e85-841729f326a5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.805961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8944f3-2d7f-486e-8e85-841729f326a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.865585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ee8944f3-2d7f-486e-8e85-841729f326a5 None None] MFA Rules not processed for user `26b01f17d3b4463fad83821410187164`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:06.867588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ee8944f3-2d7f-486e-8e85-841729f326a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:06.876923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106ef8c-ad56-4148-906b-e540157cf3e1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:06.926589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:06.956357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:06.980065 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a787ae-0356-45fb-a78b-4c67429ae0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:06.982481 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a787ae-0356-45fb-a78b-4c67429ae0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.982781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a787ae-0356-45fb-a78b-4c67429ae0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.983061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a787ae-0356-45fb-a78b-4c67429ae0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.984981 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a787ae-0356-45fb-a78b-4c67429ae0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:06.987303 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a787ae-0356-45fb-a78b-4c67429ae0c1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:07.023268 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 863/1716] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:07.025715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a787ae-0356-45fb-a78b-4c67429ae0c1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:07.028471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 854/1717] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:07.040650 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8944f3-2d7f-486e-8e85-841729f326a5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:07.042726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ee8944f3-2d7f-486e-8e85-841729f326a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:07.062265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a787ae-0356-45fb-a78b-4c67429ae0c1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:07.076712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2c69b-195a-403a-9514-c691638af5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:07.079473 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2c69b-195a-403a-9514-c691638af5b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:07.080326 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2c69b-195a-403a-9514-c691638af5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:07.080917 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2c69b-195a-403a-9514-c691638af5b9 admin admin] PATH_INFO: `/v3/projects/8057f6b1fb7b4efa883c29501728893e/users/42e59bf8ecc1402381d2260f518a6aa5/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:07.089511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2c69b-195a-403a-9514-c691638af5b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8057f6b1fb7b4efa883c29501728893e, user_id=42e59bf8ecc1402381d2260f518a6aa5, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:07.092386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2c69b-195a-403a-9514-c691638af5b9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:07.127801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 864/1718] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:07 2022] PUT /identity/v3/projects/8057f6b1fb7b4efa883c29501728893e/users/42e59bf8ecc1402381d2260f518a6aa5/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:07.138580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd507b-f0f2-482a-a928-e028839e758e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:07.138865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd507b-f0f2-482a-a928-e028839e758e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:07.139150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd507b-f0f2-482a-a928-e028839e758e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:07.231899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8944f3-2d7f-486e-8e85-841729f326a5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:07.238352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b8dd507b-f0f2-482a-a928-e028839e758e None None] MFA Rules not processed for user `42e59bf8ecc1402381d2260f518a6aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:07.255779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b8dd507b-f0f2-482a-a928-e028839e758e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:07.423615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac08165-3fd2-450f-bd4b-6f6346f9f403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:07.425133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac08165-3fd2-450f-bd4b-6f6346f9f403 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:07.425591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac08165-3fd2-450f-bd4b-6f6346f9f403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:07.426037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac08165-3fd2-450f-bd4b-6f6346f9f403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:07.429941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 865/1719] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:07 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:07.568205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac08165-3fd2-450f-bd4b-6f6346f9f403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:07.569887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac08165-3fd2-450f-bd4b-6f6346f9f403 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:07.609667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd507b-f0f2-482a-a928-e028839e758e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:07.612586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b8dd507b-f0f2-482a-a928-e028839e758e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:07.618484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 855/1720] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:07.651821 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac08165-3fd2-450f-bd4b-6f6346f9f403 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:07.673043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd507b-f0f2-482a-a928-e028839e758e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:07.699942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf4f96-b963-42a9-96c5-0033c2f8da61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:07.706489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf4f96-b963-42a9-96c5-0033c2f8da61 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:07.706489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf4f96-b963-42a9-96c5-0033c2f8da61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:07.713393 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf4f96-b963-42a9-96c5-0033c2f8da61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:07.744882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf4f96-b963-42a9-96c5-0033c2f8da61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:07.750753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf4f96-b963-42a9-96c5-0033c2f8da61 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:07.790124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 866/1721] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:07 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:07.794207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:07.818221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac08165-3fd2-450f-bd4b-6f6346f9f403 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:07.836784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc5a30-e2d6-41f1-8ed7-b4bf11ff0a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:07.837826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc5a30-e2d6-41f1-8ed7-b4bf11ff0a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:07.838131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc5a30-e2d6-41f1-8ed7-b4bf11ff0a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:07.838412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc5a30-e2d6-41f1-8ed7-b4bf11ff0a09 admin admin] PATH_INFO: `/v3/users/f5c75bc77eb94ca9bc4766aafb39eba9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:07.842415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc5a30-e2d6-41f1-8ed7-b4bf11ff0a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c75bc77eb94ca9bc4766aafb39eba9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:07.844508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc5a30-e2d6-41f1-8ed7-b4bf11ff0a09 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:07.857542 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf4f96-b963-42a9-96c5-0033c2f8da61 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:07.858228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7caf4f96-b963-42a9-96c5-0033c2f8da61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:07.895428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf4f96-b963-42a9-96c5-0033c2f8da61 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:07.914119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456dee0-66ee-40cb-81f5-e4ba6f167b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:07.918004 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456dee0-66ee-40cb-81f5-e4ba6f167b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:07.919948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456dee0-66ee-40cb-81f5-e4ba6f167b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:07.923507 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456dee0-66ee-40cb-81f5-e4ba6f167b23 admin admin] PATH_INFO: `/v3/projects/d797911a43b4435bac05c3ec29c56d8d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:07.934163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456dee0-66ee-40cb-81f5-e4ba6f167b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d797911a43b4435bac05c3ec29c56d8d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:07.946756 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456dee0-66ee-40cb-81f5-e4ba6f167b23 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:07.949979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-45cc5a30-e2d6-41f1-8ed7-b4bf11ff0a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c75bc77eb94ca9bc4766aafb39eba9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:07.950726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-45cc5a30-e2d6-41f1-8ed7-b4bf11ff0a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c75bc77eb94ca9bc4766aafb39eba9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:07.967516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-45cc5a30-e2d6-41f1-8ed7-b4bf11ff0a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c75bc77eb94ca9bc4766aafb39eba9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:07.991437 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-45cc5a30-e2d6-41f1-8ed7-b4bf11ff0a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c75bc77eb94ca9bc4766aafb39eba9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:08.000935 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-45cc5a30-e2d6-41f1-8ed7-b4bf11ff0a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c75bc77eb94ca9bc4766aafb39eba9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:08.012385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 856/1722] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:07 2022] DELETE /identity/v3/users/f5c75bc77eb94ca9bc4766aafb39eba9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:08.017285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c383e-6df5-4dba-be97-ada874db3dd6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:08.017572 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c383e-6df5-4dba-be97-ada874db3dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:08.018005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c383e-6df5-4dba-be97-ada874db3dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:08.077067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ac4c383e-6df5-4dba-be97-ada874db3dd6 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:08.078599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ac4c383e-6df5-4dba-be97-ada874db3dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:08.131597 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0456dee0-66ee-40cb-81f5-e4ba6f167b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd797911a43b4435bac05c3ec29c56d8d', 'request_id': 'req-0456dee0-66ee-40cb-81f5-e4ba6f167b23'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:08.157593 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0456dee0-66ee-40cb-81f5-e4ba6f167b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd797911a43b4435bac05c3ec29c56d8d', 'request_id': 'req-0456dee0-66ee-40cb-81f5-e4ba6f167b23'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:08.173204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0456dee0-66ee-40cb-81f5-e4ba6f167b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd797911a43b4435bac05c3ec29c56d8d', 'request_id': 'req-0456dee0-66ee-40cb-81f5-e4ba6f167b23'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:08.201240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0456dee0-66ee-40cb-81f5-e4ba6f167b23 admin admin] The token cache is being invalidate because project d797911a43b4435bac05c3ec29c56d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:08.201554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0456dee0-66ee-40cb-81f5-e4ba6f167b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:08.211694 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0456dee0-66ee-40cb-81f5-e4ba6f167b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:08.211871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 857/1723] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:08.217170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 867/1724] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:07 2022] DELETE /identity/v3/projects/d797911a43b4435bac05c3ec29c56d8d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:08.221907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c383e-6df5-4dba-be97-ada874db3dd6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:08.227440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ac4c383e-6df5-4dba-be97-ada874db3dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:08.250656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c383e-6df5-4dba-be97-ada874db3dd6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:08.269796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1505a3-e1ec-4a83-be52-487ee96b0eb2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:08.270376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1505a3-e1ec-4a83-be52-487ee96b0eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:08.271006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1505a3-e1ec-4a83-be52-487ee96b0eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:08.275119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:08.275868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:08.276193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:08.280632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] PATH_INFO: `/v3/users/6a6bbeba0fe247949ffa22d24e8ff429` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:08.283538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a6bbeba0fe247949ffa22d24e8ff429)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:08.285663 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:08.303144 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:08.314488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9a1505a3-e1ec-4a83-be52-487ee96b0eb2 None None] MFA Rules not processed for user `df7d511abd8544878e7d0097a1ab0319`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:08.316930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9a1505a3-e1ec-4a83-be52-487ee96b0eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:08.410217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a6bbeba0fe247949ffa22d24e8ff429'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:08.413222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a6bbeba0fe247949ffa22d24e8ff429'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:08.434053 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a6bbeba0fe247949ffa22d24e8ff429'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:08.443960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:08.451621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a6bbeba0fe247949ffa22d24e8ff429'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:08.461825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a6bbeba0fe247949ffa22d24e8ff429'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:08.470023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 858/1725] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:08 2022] DELETE /identity/v3/users/6a6bbeba0fe247949ffa22d24e8ff429 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:08.472049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:08.474039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:08.515722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 868/1726] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:08.534318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1505a3-e1ec-4a83-be52-487ee96b0eb2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:08.537763 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9a1505a3-e1ec-4a83-be52-487ee96b0eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:08.650428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f20874-53d3-4ee3-a515-b9ce36b05802 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:08.662215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a202418-44c7-4c3f-9b11-526843c63abd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:08.663262 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a202418-44c7-4c3f-9b11-526843c63abd service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:08.663649 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a202418-44c7-4c3f-9b11-526843c63abd service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:08.663846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a202418-44c7-4c3f-9b11-526843c63abd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:08.668692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3a202418-44c7-4c3f-9b11-526843c63abd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:08.801522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1505a3-e1ec-4a83-be52-487ee96b0eb2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:08.892513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a202418-44c7-4c3f-9b11-526843c63abd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:08.895455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a202418-44c7-4c3f-9b11-526843c63abd service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:08.910349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 859/1727] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:08.961159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750cfba9-34e7-4a53-ac35-c975a8658d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:08.964263 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cfba9-34e7-4a53-ac35-c975a8658d77 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:08.964759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cfba9-34e7-4a53-ac35-c975a8658d77 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:08.965697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cfba9-34e7-4a53-ac35-c975a8658d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:09.072029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:09.143166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750cfba9-34e7-4a53-ac35-c975a8658d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:09.146623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750cfba9-34e7-4a53-ac35-c975a8658d77 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:09.172889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 869/1728] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:09.216397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a202418-44c7-4c3f-9b11-526843c63abd service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:09.218657 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3a202418-44c7-4c3f-9b11-526843c63abd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:09.240601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a202418-44c7-4c3f-9b11-526843c63abd service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:09.260698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:09.262893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:09.263389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:09.263894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3 admin admin] PATH_INFO: `/v3/projects/b70212ee7b494d638957cd5771b14f18` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:09.266201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b70212ee7b494d638957cd5771b14f18)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:09.269023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:09.418523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b70212ee7b494d638957cd5771b14f18', 'request_id': 'req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:09.436364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b70212ee7b494d638957cd5771b14f18', 'request_id': 'req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:09.467130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b70212ee7b494d638957cd5771b14f18', 'request_id': 'req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:09.489031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3 admin admin] The token cache is being invalidate because project b70212ee7b494d638957cd5771b14f18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:09.489196 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:09.489880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1fb6e89a-66fe-4fbf-98d9-8a5dc72007b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:09.491973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 860/1729] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:09 2022] DELETE /identity/v3/projects/b70212ee7b494d638957cd5771b14f18 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:09.496828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367e427-7c0f-4188-954e-c4c666cfc5d6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:09.497248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367e427-7c0f-4188-954e-c4c666cfc5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:09.497487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367e427-7c0f-4188-954e-c4c666cfc5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:09.542038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9367e427-7c0f-4188-954e-c4c666cfc5d6 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:09.545261 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9367e427-7c0f-4188-954e-c4c666cfc5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:09.686709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 870/1730] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:09.695781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfa28c-b3c4-447b-84e1-8fe7c7438aee None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:09.696074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfa28c-b3c4-447b-84e1-8fe7c7438aee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:09.696358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfa28c-b3c4-447b-84e1-8fe7c7438aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:09.748860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-18bfa28c-b3c4-447b-84e1-8fe7c7438aee None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:09.750132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-18bfa28c-b3c4-447b-84e1-8fe7c7438aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:09.771459 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 861/1731] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:09.776940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367e427-7c0f-4188-954e-c4c666cfc5d6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:09.801422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367e427-7c0f-4188-954e-c4c666cfc5d6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:09.829815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fbc97-d983-4622-9c09-92614b253b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:09.830525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fbc97-d983-4622-9c09-92614b253b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:09.831752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fbc97-d983-4622-9c09-92614b253b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:09.832043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fbc97-d983-4622-9c09-92614b253b36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:09.835328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08fbc97-d983-4622-9c09-92614b253b36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:09.836181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08fbc97-d983-4622-9c09-92614b253b36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:09.843198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08fbc97-d983-4622-9c09-92614b253b36 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:09.891494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 871/1732] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:09.895879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bfa28c-b3c4-447b-84e1-8fe7c7438aee None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:09.908889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bfa28c-b3c4-447b-84e1-8fe7c7438aee None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:09.933277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a05ba1-ede9-4bd8-a1f5-bf65e45f314c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:09.933900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a05ba1-ede9-4bd8-a1f5-bf65e45f314c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:09.934179 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a05ba1-ede9-4bd8-a1f5-bf65e45f314c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:09.934429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a05ba1-ede9-4bd8-a1f5-bf65e45f314c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:09.935140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a05ba1-ede9-4bd8-a1f5-bf65e45f314c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:09.941905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a05ba1-ede9-4bd8-a1f5-bf65e45f314c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:09.952991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:10.000283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 862/1733] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:09 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:10.004214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fbc97-d983-4622-9c09-92614b253b36 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:10.018703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fbc97-d983-4622-9c09-92614b253b36 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:10.171744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679d170-d670-479b-96c2-106e47e70bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:10.173588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679d170-d670-479b-96c2-106e47e70bcd admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:10.173887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679d170-d670-479b-96c2-106e47e70bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:10.174177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679d170-d670-479b-96c2-106e47e70bcd admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:10.176214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679d170-d670-479b-96c2-106e47e70bcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:10.180064 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679d170-d670-479b-96c2-106e47e70bcd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:10.222282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 872/1734] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:10 2022] POST /identity/v3/users => generated 282 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:10.227952 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a05ba1-ede9-4bd8-a1f5-bf65e45f314c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:10.245708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a05ba1-ede9-4bd8-a1f5-bf65e45f314c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:10.264916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85257d-45f3-4003-89f7-fd703adb16bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:10.265616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85257d-45f3-4003-89f7-fd703adb16bd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:10.265901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85257d-45f3-4003-89f7-fd703adb16bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:10.266175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85257d-45f3-4003-89f7-fd703adb16bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:10.266912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85257d-45f3-4003-89f7-fd703adb16bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:10.268680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85257d-45f3-4003-89f7-fd703adb16bd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:10.279957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679d170-d670-479b-96c2-106e47e70bcd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:10.282846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1679d170-d670-479b-96c2-106e47e70bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:10.294746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 863/1735] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:10.298190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85257d-45f3-4003-89f7-fd703adb16bd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:10.305512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679d170-d670-479b-96c2-106e47e70bcd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:10.313158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5229adb-d1c8-4162-941f-cf0bab4bcee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:10.314581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5229adb-d1c8-4162-941f-cf0bab4bcee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:10.315098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5229adb-d1c8-4162-941f-cf0bab4bcee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:10.315615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5229adb-d1c8-4162-941f-cf0bab4bcee7 admin admin] PATH_INFO: `/v3/users/f55347c643984fb6a4a8655ad992862d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:10.317542 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5229adb-d1c8-4162-941f-cf0bab4bcee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f55347c643984fb6a4a8655ad992862d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:10.317670 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85257d-45f3-4003-89f7-fd703adb16bd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:10.320170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5229adb-d1c8-4162-941f-cf0bab4bcee7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:10.330701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f4da5-7594-47bf-b836-98052c8b8a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:10.331531 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f4da5-7594-47bf-b836-98052c8b8a46 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:10.331851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f4da5-7594-47bf-b836-98052c8b8a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:10.332128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f4da5-7594-47bf-b836-98052c8b8a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:10.332873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f4da5-7594-47bf-b836-98052c8b8a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:10.334488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f4da5-7594-47bf-b836-98052c8b8a46 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:10.368288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 864/1736] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:10.369074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f4da5-7594-47bf-b836-98052c8b8a46 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:10.392411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f4da5-7594-47bf-b836-98052c8b8a46 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:10.404487 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef71a45-73a1-4683-bf45-6820037ce036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:10.405389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef71a45-73a1-4683-bf45-6820037ce036 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:10.405709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef71a45-73a1-4683-bf45-6820037ce036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:10.405946 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef71a45-73a1-4683-bf45-6820037ce036 admin admin] PATH_INFO: `/v3/projects/7718f93df9bb4e1d8312a606d25d7bf3/users/73a1bf83e2f640328ed4fefe3dc0018b/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:10.433208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5229adb-d1c8-4162-941f-cf0bab4bcee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f55347c643984fb6a4a8655ad992862d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:10.434902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5229adb-d1c8-4162-941f-cf0bab4bcee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f55347c643984fb6a4a8655ad992862d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:10.436041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5229adb-d1c8-4162-941f-cf0bab4bcee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f55347c643984fb6a4a8655ad992862d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:10.437717 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:10.445330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef71a45-73a1-4683-bf45-6820037ce036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7718f93df9bb4e1d8312a606d25d7bf3, user_id=73a1bf83e2f640328ed4fefe3dc0018b, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:10.449297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef71a45-73a1-4683-bf45-6820037ce036 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:10.462157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5229adb-d1c8-4162-941f-cf0bab4bcee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f55347c643984fb6a4a8655ad992862d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:10.473711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 865/1737] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:10 2022] PUT /identity/v3/projects/7718f93df9bb4e1d8312a606d25d7bf3/users/73a1bf83e2f640328ed4fefe3dc0018b/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:10.484456 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91adf53-96f8-4205-86b4-96405fa7bbb0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:10.485098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91adf53-96f8-4205-86b4-96405fa7bbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:10.485577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91adf53-96f8-4205-86b4-96405fa7bbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:10.505183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5229adb-d1c8-4162-941f-cf0bab4bcee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f55347c643984fb6a4a8655ad992862d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:10.522337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b91adf53-96f8-4205-86b4-96405fa7bbb0 None None] MFA Rules not processed for user `73a1bf83e2f640328ed4fefe3dc0018b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:10.526089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b91adf53-96f8-4205-86b4-96405fa7bbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:10.537356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 873/1738] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:10 2022] DELETE /identity/v3/users/f55347c643984fb6a4a8655ad992862d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:10.539276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49bc86-c127-4a7f-be77-3923d3974cd8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:10.539576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49bc86-c127-4a7f-be77-3923d3974cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:10.540020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49bc86-c127-4a7f-be77-3923d3974cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:10.579621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8a49bc86-c127-4a7f-be77-3923d3974cd8 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:10.582801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8a49bc86-c127-4a7f-be77-3923d3974cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:10.679002 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 866/1739] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:10 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:10.684626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91adf53-96f8-4205-86b4-96405fa7bbb0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:10.691572 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b91adf53-96f8-4205-86b4-96405fa7bbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:10.722110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 874/1740] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:10.774307 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888ace3-7d71-4748-9961-deb2c110bacd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:10.774856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888ace3-7d71-4748-9961-deb2c110bacd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:10.775148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888ace3-7d71-4748-9961-deb2c110bacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:10.823425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3888ace3-7d71-4748-9961-deb2c110bacd None None] MFA Rules not processed for user `f6390dc942db430e8dd648901900432b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:10.825269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3888ace3-7d71-4748-9961-deb2c110bacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:10.883103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91adf53-96f8-4205-86b4-96405fa7bbb0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:10.898687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997ea0c-8b30-45f1-8051-3d9120f36809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:10.899388 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997ea0c-8b30-45f1-8051-3d9120f36809 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:10.899693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997ea0c-8b30-45f1-8051-3d9120f36809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:10.899964 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997ea0c-8b30-45f1-8051-3d9120f36809 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:10.900675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997ea0c-8b30-45f1-8051-3d9120f36809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:10.908153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997ea0c-8b30-45f1-8051-3d9120f36809 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:10.982945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 867/1741] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:10 2022] POST /identity/v3/projects => generated 363 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:10.984133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997ea0c-8b30-45f1-8051-3d9120f36809 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:10.989139 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6997ea0c-8b30-45f1-8051-3d9120f36809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:11.007817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 875/1742] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:11.008788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888ace3-7d71-4748-9961-deb2c110bacd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:11.011902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3888ace3-7d71-4748-9961-deb2c110bacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:11.057877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:11.180492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997ea0c-8b30-45f1-8051-3d9120f36809 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:11.192415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7142b7-b980-4adb-866e-da5a00caf0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:11.193243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7142b7-b980-4adb-866e-da5a00caf0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:11.193526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7142b7-b980-4adb-866e-da5a00caf0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:11.193839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7142b7-b980-4adb-866e-da5a00caf0be admin admin] PATH_INFO: `/v3/users/da4f35033ad54f799c3727c7f83cffd1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:11.195720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7142b7-b980-4adb-866e-da5a00caf0be admin admin] RBAC: Authorizing `identity:delete_user(user_id=da4f35033ad54f799c3727c7f83cffd1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:11.197687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7142b7-b980-4adb-866e-da5a00caf0be admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:11.225878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888ace3-7d71-4748-9961-deb2c110bacd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:11.243343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:11.353433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4c7142b7-b980-4adb-866e-da5a00caf0be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da4f35033ad54f799c3727c7f83cffd1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:11.356223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4c7142b7-b980-4adb-866e-da5a00caf0be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da4f35033ad54f799c3727c7f83cffd1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:11.361391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6386ac-ea98-44d7-847a-fb41cd7c8c7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:11.363604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6386ac-ea98-44d7-847a-fb41cd7c8c7a service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:11.364064 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6386ac-ea98-44d7-847a-fb41cd7c8c7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:11.365545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6386ac-ea98-44d7-847a-fb41cd7c8c7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:11.375064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4c7142b7-b980-4adb-866e-da5a00caf0be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da4f35033ad54f799c3727c7f83cffd1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:11.390866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6386ac-ea98-44d7-847a-fb41cd7c8c7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:11.392589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6386ac-ea98-44d7-847a-fb41cd7c8c7a service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:11.399076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4c7142b7-b980-4adb-866e-da5a00caf0be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da4f35033ad54f799c3727c7f83cffd1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:11.427508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 876/1743] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:11.429849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6386ac-ea98-44d7-847a-fb41cd7c8c7a service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:11.430548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4c7142b7-b980-4adb-866e-da5a00caf0be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da4f35033ad54f799c3727c7f83cffd1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:11.442033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 868/1744] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:10 2022] DELETE /identity/v3/users/da4f35033ad54f799c3727c7f83cffd1 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:11.443969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8fe71-a476-45fc-86ae-19c6fc1b7b8a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:11.446410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8fe71-a476-45fc-86ae-19c6fc1b7b8a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:11.446913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8fe71-a476-45fc-86ae-19c6fc1b7b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:11.453718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6386ac-ea98-44d7-847a-fb41cd7c8c7a service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:11.490723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-81a8fe71-a476-45fc-86ae-19c6fc1b7b8a None None] MFA Rules not processed for user `f6390dc942db430e8dd648901900432b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:11.497266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-81a8fe71-a476-45fc-86ae-19c6fc1b7b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:11.618903 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e87d0-5e48-442e-8392-4d35edcf8a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:11.622650 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e87d0-5e48-442e-8392-4d35edcf8a3e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:11.623086 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e87d0-5e48-442e-8392-4d35edcf8a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:11.624237 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e87d0-5e48-442e-8392-4d35edcf8a3e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:11.625611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e87d0-5e48-442e-8392-4d35edcf8a3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:11.631981 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e87d0-5e48-442e-8392-4d35edcf8a3e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:11.690350 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 869/1745] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:11.692204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8fe71-a476-45fc-86ae-19c6fc1b7b8a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:11.697401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-81a8fe71-a476-45fc-86ae-19c6fc1b7b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:11.729669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 877/1746] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:11 2022] POST /identity/v3/users => generated 288 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:11.741144 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05713f-8b11-4779-b23b-2a372a842a15 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:11.742445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05713f-8b11-4779-b23b-2a372a842a15 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:11.742923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05713f-8b11-4779-b23b-2a372a842a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:11.752106 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8fe71-a476-45fc-86ae-19c6fc1b7b8a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:11.762996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:11.763995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:11.764278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:11.764555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] PATH_INFO: `/v3/projects/73388109969f4bb9b1ba7b5e61edb813` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:11.765776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=73388109969f4bb9b1ba7b5e61edb813)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:11.769672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:11.792645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0f05713f-8b11-4779-b23b-2a372a842a15 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:11.795788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0f05713f-8b11-4779-b23b-2a372a842a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:11.858565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:11.881669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 878/1747] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:11.883038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05713f-8b11-4779-b23b-2a372a842a15 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:11.890428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0f05713f-8b11-4779-b23b-2a372a842a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:11.896989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73388109969f4bb9b1ba7b5e61edb813', 'request_id': 'req-db5592a5-c748-4765-bf56-960c65ec1a5e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:11.915436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73388109969f4bb9b1ba7b5e61edb813', 'request_id': 'req-db5592a5-c748-4765-bf56-960c65ec1a5e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:11.937552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73388109969f4bb9b1ba7b5e61edb813', 'request_id': 'req-db5592a5-c748-4765-bf56-960c65ec1a5e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:11.978470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] The token cache is being invalidate because project 73388109969f4bb9b1ba7b5e61edb813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:11.978794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:11.980691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:11.990651 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 870/1748] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:11 2022] DELETE /identity/v3/projects/73388109969f4bb9b1ba7b5e61edb813 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:11.995333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:11.999258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:12.245295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5592a5-c748-4765-bf56-960c65ec1a5e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:12.256652 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fa6e7-fb8a-4d67-841f-6a5db2bd5aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:12.257901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fa6e7-fb8a-4d67-841f-6a5db2bd5aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:12.258189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fa6e7-fb8a-4d67-841f-6a5db2bd5aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:12.258469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fa6e7-fb8a-4d67-841f-6a5db2bd5aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:12.259217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fa6e7-fb8a-4d67-841f-6a5db2bd5aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:12.260961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fa6e7-fb8a-4d67-841f-6a5db2bd5aa6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:12.280822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05713f-8b11-4779-b23b-2a372a842a15 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:12.303337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 871/1749] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:12.315908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fa6e7-fb8a-4d67-841f-6a5db2bd5aa6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:12.317973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-288fa6e7-fb8a-4d67-841f-6a5db2bd5aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:12.440101 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:12.479317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fa6e7-fb8a-4d67-841f-6a5db2bd5aa6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:12.491924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9f88d-5bab-4024-8171-0bf71ec93d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:12.492901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9f88d-5bab-4024-8171-0bf71ec93d16 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:12.493365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9f88d-5bab-4024-8171-0bf71ec93d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:12.494208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9f88d-5bab-4024-8171-0bf71ec93d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:12.497063 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cdf9f88d-5bab-4024-8171-0bf71ec93d16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:12.537422 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9f88d-5bab-4024-8171-0bf71ec93d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:12.538928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9f88d-5bab-4024-8171-0bf71ec93d16 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:12.560309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3902ab-1ac8-47e3-b120-feb7b1e71b78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:12.560987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3902ab-1ac8-47e3-b120-feb7b1e71b78 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:12.561272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3902ab-1ac8-47e3-b120-feb7b1e71b78 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:12.561557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3902ab-1ac8-47e3-b120-feb7b1e71b78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:12.565371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4b3902ab-1ac8-47e3-b120-feb7b1e71b78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:12.574812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 872/1750] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:12.575997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9f88d-5bab-4024-8171-0bf71ec93d16 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:12.582053 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cdf9f88d-5bab-4024-8171-0bf71ec93d16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:12.620806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9f88d-5bab-4024-8171-0bf71ec93d16 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:12.636244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:12.637424 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:12.637865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:12.638297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] PATH_INFO: `/v3/users/26b01f17d3b4463fad83821410187164` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:12.640986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26b01f17d3b4463fad83821410187164)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:12.643374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:12.718636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3902ab-1ac8-47e3-b120-feb7b1e71b78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:12.720462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3902ab-1ac8-47e3-b120-feb7b1e71b78 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:12.746872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 879/1751] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 862 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:12.748103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26b01f17d3b4463fad83821410187164'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:12.748758 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3902ab-1ac8-47e3-b120-feb7b1e71b78 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:12.749544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26b01f17d3b4463fad83821410187164'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:12.766762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26b01f17d3b4463fad83821410187164'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:12.768490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:12.773819 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3902ab-1ac8-47e3-b120-feb7b1e71b78 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:12.779923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3797ca3-38e3-41ad-8125-86a56ac9f564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:12.781575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3797ca3-38e3-41ad-8125-86a56ac9f564 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:12.782094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3797ca3-38e3-41ad-8125-86a56ac9f564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:12.782194 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3797ca3-38e3-41ad-8125-86a56ac9f564 admin admin] PATH_INFO: `/v3/projects/eb530f6181524731875b76a146ab1d45/users/e10ceb6dd39a4cdd99adaab248dcdf8e/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:12.801755 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3797ca3-38e3-41ad-8125-86a56ac9f564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb530f6181524731875b76a146ab1d45, user_id=e10ceb6dd39a4cdd99adaab248dcdf8e, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:12.807914 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3797ca3-38e3-41ad-8125-86a56ac9f564 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:12.847721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26b01f17d3b4463fad83821410187164'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:12.856257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 880/1752] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:12 2022] PUT /identity/v3/projects/eb530f6181524731875b76a146ab1d45/users/e10ceb6dd39a4cdd99adaab248dcdf8e/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:12.857227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3797ca3-38e3-41ad-8125-86a56ac9f564 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:12.890479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26b01f17d3b4463fad83821410187164'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:12.912985 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 873/1753] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:12 2022] DELETE /identity/v3/users/26b01f17d3b4463fad83821410187164 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:13.066331 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3797ca3-38e3-41ad-8125-86a56ac9f564 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:13.257667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:13.274910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e93a3-43f3-4a33-a1d0-542ec2b4421d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:13.276549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e93a3-43f3-4a33-a1d0-542ec2b4421d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:13.276834 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e93a3-43f3-4a33-a1d0-542ec2b4421d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:13.277108 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e93a3-43f3-4a33-a1d0-542ec2b4421d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:13.278426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e93a3-43f3-4a33-a1d0-542ec2b4421d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:13.281679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e93a3-43f3-4a33-a1d0-542ec2b4421d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:13.307081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 881/1754] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:13.309273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:13.310901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:13.334801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287bacc-546b-4760-b89b-530dc0509f74 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:13.345595 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffcd52-6ba3-43af-9cd1-2d347c191442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:13.346811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffcd52-6ba3-43af-9cd1-2d347c191442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:13.347373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffcd52-6ba3-43af-9cd1-2d347c191442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:13.347932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffcd52-6ba3-43af-9cd1-2d347c191442 admin admin] PATH_INFO: `/v3/projects/eb530f6181524731875b76a146ab1d45/users/e10ceb6dd39a4cdd99adaab248dcdf8e/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:13.354891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffcd52-6ba3-43af-9cd1-2d347c191442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb530f6181524731875b76a146ab1d45, user_id=e10ceb6dd39a4cdd99adaab248dcdf8e, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:13.363016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffcd52-6ba3-43af-9cd1-2d347c191442 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:13.394909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 874/1755] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:13 2022] PUT /identity/v3/projects/eb530f6181524731875b76a146ab1d45/users/e10ceb6dd39a4cdd99adaab248dcdf8e/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:13.407885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17948f88-d112-4ec9-a9b8-2aa36adef993 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:13.409380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17948f88-d112-4ec9-a9b8-2aa36adef993 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:13.409755 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17948f88-d112-4ec9-a9b8-2aa36adef993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:13.474282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-17948f88-d112-4ec9-a9b8-2aa36adef993 None None] MFA Rules not processed for user `e10ceb6dd39a4cdd99adaab248dcdf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:13.476238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-17948f88-d112-4ec9-a9b8-2aa36adef993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:13.689350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 882/1756] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:13 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:13.702765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffcd52-6ba3-43af-9cd1-2d347c191442 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:13.707308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3dffcd52-6ba3-43af-9cd1-2d347c191442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:13.818143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffcd52-6ba3-43af-9cd1-2d347c191442 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:13.823933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95af73-89d9-4d0d-8732-bce1812a4c46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:13.824616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95af73-89d9-4d0d-8732-bce1812a4c46 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:13.825896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95af73-89d9-4d0d-8732-bce1812a4c46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:13.826228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95af73-89d9-4d0d-8732-bce1812a4c46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:13.870271 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95af73-89d9-4d0d-8732-bce1812a4c46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:13.878584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95af73-89d9-4d0d-8732-bce1812a4c46 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:13.910126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 875/1757] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:13 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:13.937287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23671575-51b8-44e6-8ed0-a888e1312c5d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:13.937807 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23671575-51b8-44e6-8ed0-a888e1312c5d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:13.938022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23671575-51b8-44e6-8ed0-a888e1312c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:13.978967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-23671575-51b8-44e6-8ed0-a888e1312c5d None None] MFA Rules not processed for user `df7d511abd8544878e7d0097a1ab0319`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:13.980239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-23671575-51b8-44e6-8ed0-a888e1312c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:14.026207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95af73-89d9-4d0d-8732-bce1812a4c46 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:14.029090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ec95af73-89d9-4d0d-8732-bce1812a4c46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:14.148990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 883/1758] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:14.155711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23671575-51b8-44e6-8ed0-a888e1312c5d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:14.172538 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23671575-51b8-44e6-8ed0-a888e1312c5d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:14.183379 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d799b-d422-44b2-9da4-b0c82c8caa71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:14.184929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d799b-d422-44b2-9da4-b0c82c8caa71 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:14.186951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d799b-d422-44b2-9da4-b0c82c8caa71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:14.187321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d799b-d422-44b2-9da4-b0c82c8caa71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:14.193862 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fb2d799b-d422-44b2-9da4-b0c82c8caa71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:14.227445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95af73-89d9-4d0d-8732-bce1812a4c46 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:14.233602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:14.236496 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:14.237541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:14.238326 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] PATH_INFO: `/v3/projects/87306ef18d364b5fabb1687ce5ca3280` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:14.239847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87306ef18d364b5fabb1687ce5ca3280)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:14.249802 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:14.335465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87306ef18d364b5fabb1687ce5ca3280', 'request_id': 'req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.347239 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87306ef18d364b5fabb1687ce5ca3280', 'request_id': 'req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.358523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:14.361815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87306ef18d364b5fabb1687ce5ca3280', 'request_id': 'req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.390795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d799b-d422-44b2-9da4-b0c82c8caa71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:14.394995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d799b-d422-44b2-9da4-b0c82c8caa71 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:14.401871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] The token cache is being invalidate because project 87306ef18d364b5fabb1687ce5ca3280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:14.402169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.412800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.416983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 876/1759] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:14 2022] DELETE /identity/v3/projects/87306ef18d364b5fabb1687ce5ca3280 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:14.417832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:14.428803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:14.437681 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 884/1760] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:14 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:14.439472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d799b-d422-44b2-9da4-b0c82c8caa71 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:14.467071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d799b-d422-44b2-9da4-b0c82c8caa71 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:14.500350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:14.660352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fe26e-c66c-4a17-a629-7f5716b03db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:14.664674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fe26e-c66c-4a17-a629-7f5716b03db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:14.664962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fe26e-c66c-4a17-a629-7f5716b03db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:14.665243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fe26e-c66c-4a17-a629-7f5716b03db8 admin admin] PATH_INFO: `/v3/users/42e59bf8ecc1402381d2260f518a6aa5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:14.666457 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce186e-dfc8-492d-a1cb-4f6c6e741c55 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:14.668849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68fe26e-c66c-4a17-a629-7f5716b03db8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42e59bf8ecc1402381d2260f518a6aa5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:14.673820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf51add-333d-475f-b52e-0fb863bf964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:14.674711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68fe26e-c66c-4a17-a629-7f5716b03db8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:14.674870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf51add-333d-475f-b52e-0fb863bf964d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:14.675254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf51add-333d-475f-b52e-0fb863bf964d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:14.675631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf51add-333d-475f-b52e-0fb863bf964d admin admin] PATH_INFO: `/v3/projects/a1136b2daaba4d58aae0024c215d2843` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:14.677436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf51add-333d-475f-b52e-0fb863bf964d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1136b2daaba4d58aae0024c215d2843)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:14.679480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf51add-333d-475f-b52e-0fb863bf964d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:14.790610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4cf51add-333d-475f-b52e-0fb863bf964d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1136b2daaba4d58aae0024c215d2843', 'request_id': 'req-4cf51add-333d-475f-b52e-0fb863bf964d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.795143 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e68fe26e-c66c-4a17-a629-7f5716b03db8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42e59bf8ecc1402381d2260f518a6aa5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.795467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e68fe26e-c66c-4a17-a629-7f5716b03db8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42e59bf8ecc1402381d2260f518a6aa5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.796500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e68fe26e-c66c-4a17-a629-7f5716b03db8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42e59bf8ecc1402381d2260f518a6aa5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.810690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4cf51add-333d-475f-b52e-0fb863bf964d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1136b2daaba4d58aae0024c215d2843', 'request_id': 'req-4cf51add-333d-475f-b52e-0fb863bf964d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.820441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4cf51add-333d-475f-b52e-0fb863bf964d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1136b2daaba4d58aae0024c215d2843', 'request_id': 'req-4cf51add-333d-475f-b52e-0fb863bf964d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.834183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e68fe26e-c66c-4a17-a629-7f5716b03db8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42e59bf8ecc1402381d2260f518a6aa5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.844517 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4cf51add-333d-475f-b52e-0fb863bf964d admin admin] The token cache is being invalidate because project a1136b2daaba4d58aae0024c215d2843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:14.844814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4cf51add-333d-475f-b52e-0fb863bf964d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.846596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4cf51add-333d-475f-b52e-0fb863bf964d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.847115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e68fe26e-c66c-4a17-a629-7f5716b03db8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42e59bf8ecc1402381d2260f518a6aa5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:14.848608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 877/1761] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:14 2022] DELETE /identity/v3/projects/a1136b2daaba4d58aae0024c215d2843 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:14.850676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d28010-feca-4f76-ada7-72354a0a6227 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:14.851108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d28010-feca-4f76-ada7-72354a0a6227 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:14.851597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d28010-feca-4f76-ada7-72354a0a6227 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:14.854365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 878/1762] 10.222.0.65 () {54 vars in 912 bytes} [Thu Jun 23 11:02:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:14.861217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d28010-feca-4f76-ada7-72354a0a6227 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:14.863724 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-90d28010-feca-4f76-ada7-72354a0a6227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:14.871571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 885/1763] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:14 2022] DELETE /identity/v3/users/42e59bf8ecc1402381d2260f518a6aa5 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:14.872915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de18a32-d5f5-45b8-9ede-bbce2c81f389 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:14.873204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de18a32-d5f5-45b8-9ede-bbce2c81f389 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:14.873479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de18a32-d5f5-45b8-9ede-bbce2c81f389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:14.936522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5de18a32-d5f5-45b8-9ede-bbce2c81f389 None None] MFA Rules not processed for user `303e8aae1bd04228bedb145dec891a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:14.938660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5de18a32-d5f5-45b8-9ede-bbce2c81f389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:15.035217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d28010-feca-4f76-ada7-72354a0a6227 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:15.064634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b0ac6-6304-4de6-9284-af80959a3517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:15.065701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b0ac6-6304-4de6-9284-af80959a3517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:15.066092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b0ac6-6304-4de6-9284-af80959a3517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:15.066477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b0ac6-6304-4de6-9284-af80959a3517 admin admin] PATH_INFO: `/v3/projects/ccf2ef30c6f84aafb066cfa08c397d05` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:15.070326 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b0ac6-6304-4de6-9284-af80959a3517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf2ef30c6f84aafb066cfa08c397d05)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:15.073191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b0ac6-6304-4de6-9284-af80959a3517 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:15.163734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:15.211811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6b5b0ac6-6304-4de6-9284-af80959a3517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf2ef30c6f84aafb066cfa08c397d05', 'request_id': 'req-6b5b0ac6-6304-4de6-9284-af80959a3517'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:15.226887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6b5b0ac6-6304-4de6-9284-af80959a3517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf2ef30c6f84aafb066cfa08c397d05', 'request_id': 'req-6b5b0ac6-6304-4de6-9284-af80959a3517'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:15.239042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6b5b0ac6-6304-4de6-9284-af80959a3517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf2ef30c6f84aafb066cfa08c397d05', 'request_id': 'req-6b5b0ac6-6304-4de6-9284-af80959a3517'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:15.292804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 886/1764] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:15.292804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fd8cb-57e2-4b69-ab5b-ec0e48b4ff8a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:15.292804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fd8cb-57e2-4b69-ab5b-ec0e48b4ff8a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:15.292804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fd8cb-57e2-4b69-ab5b-ec0e48b4ff8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:15.303740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6b5b0ac6-6304-4de6-9284-af80959a3517 admin admin] The token cache is being invalidate because project ccf2ef30c6f84aafb066cfa08c397d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:15.304840 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6b5b0ac6-6304-4de6-9284-af80959a3517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:15.309257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6b5b0ac6-6304-4de6-9284-af80959a3517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:15.312069 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 879/1765] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:14 2022] DELETE /identity/v3/projects/ccf2ef30c6f84aafb066cfa08c397d05 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:15.313720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99d025-c2d6-41fe-bebd-a9d3d3136745 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:15.314099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99d025-c2d6-41fe-bebd-a9d3d3136745 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:15.314482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99d025-c2d6-41fe-bebd-a9d3d3136745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:15.327381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-af0fd8cb-57e2-4b69-ab5b-ec0e48b4ff8a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:15.329080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af0fd8cb-57e2-4b69-ab5b-ec0e48b4ff8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:15.395333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:15.440999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1e99d025-c2d6-41fe-bebd-a9d3d3136745 None None] MFA Rules not processed for user `e96b136657864104a51900debd1fbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:15.442243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1e99d025-c2d6-41fe-bebd-a9d3d3136745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:15.479039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 887/1766] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:15.480927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfc97d-f903-412e-ac12-e5d3b3953d06 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:15.481224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfc97d-f903-412e-ac12-e5d3b3953d06 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:15.481506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfc97d-f903-412e-ac12-e5d3b3953d06 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:15.486589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 888/1767] 10.222.0.65 () {54 vars in 912 bytes} [Thu Jun 23 11:02:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:15.488959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfc97d-f903-412e-ac12-e5d3b3953d06 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:15.491084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d4bfc97d-f903-412e-ac12-e5d3b3953d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:15.528683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfc97d-f903-412e-ac12-e5d3b3953d06 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:15.534212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135adf64-104b-4fc1-9502-1fe3f6de062a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:15.535363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135adf64-104b-4fc1-9502-1fe3f6de062a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:15.535925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135adf64-104b-4fc1-9502-1fe3f6de062a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:15.536375 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135adf64-104b-4fc1-9502-1fe3f6de062a admin admin] PATH_INFO: `/v3/users/f9cb2b5c72f74db9aac85848eb5dcf13` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:15.538846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135adf64-104b-4fc1-9502-1fe3f6de062a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9cb2b5c72f74db9aac85848eb5dcf13)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:15.542332 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135adf64-104b-4fc1-9502-1fe3f6de062a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:15.574889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 880/1768] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:02:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:15.580282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d2251-1c67-4e72-a9af-2c962bc409b1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:15.580655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d2251-1c67-4e72-a9af-2c962bc409b1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:15.580963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d2251-1c67-4e72-a9af-2c962bc409b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:15.629069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-135adf64-104b-4fc1-9502-1fe3f6de062a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9cb2b5c72f74db9aac85848eb5dcf13'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:15.630060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-135adf64-104b-4fc1-9502-1fe3f6de062a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9cb2b5c72f74db9aac85848eb5dcf13'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:15.632304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-135adf64-104b-4fc1-9502-1fe3f6de062a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9cb2b5c72f74db9aac85848eb5dcf13'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:15.649679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fa4d2251-1c67-4e72-a9af-2c962bc409b1 None None] MFA Rules not processed for user `6bb5a1f18dff4395bec9c205a9937493`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:15.650099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fa4d2251-1c67-4e72-a9af-2c962bc409b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:15.682825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-135adf64-104b-4fc1-9502-1fe3f6de062a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9cb2b5c72f74db9aac85848eb5dcf13'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:15.697417 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-135adf64-104b-4fc1-9502-1fe3f6de062a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9cb2b5c72f74db9aac85848eb5dcf13'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:15.722515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 889/1769] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:15 2022] DELETE /identity/v3/users/f9cb2b5c72f74db9aac85848eb5dcf13 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:15.725151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0d19a-82b1-475e-bb50-622dd2104139 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:15.725443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0d19a-82b1-475e-bb50-622dd2104139 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:15.730395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0d19a-82b1-475e-bb50-622dd2104139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:15.770981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 881/1770] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:15.772999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592966a-d4ea-4093-8806-f4d47361bd10 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:15.774154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592966a-d4ea-4093-8806-f4d47361bd10 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:15.774650 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592966a-d4ea-4093-8806-f4d47361bd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:15.781950 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d4e0d19a-82b1-475e-bb50-622dd2104139 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:15.783230 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d4e0d19a-82b1-475e-bb50-622dd2104139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:15.809311 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3592966a-d4ea-4093-8806-f4d47361bd10 None None] MFA Rules not processed for user `e96b136657864104a51900debd1fbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:15.812163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3592966a-d4ea-4093-8806-f4d47361bd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:15.905977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:15.967528 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 882/1771] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:02:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:15.969247 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592966a-d4ea-4093-8806-f4d47361bd10 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:15.971026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3592966a-d4ea-4093-8806-f4d47361bd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:16.030367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592966a-d4ea-4093-8806-f4d47361bd10 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:16.034966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23003181-00d1-4369-bb29-6fd3a3a786ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:16.036099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23003181-00d1-4369-bb29-6fd3a3a786ee service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:16.036450 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23003181-00d1-4369-bb29-6fd3a3a786ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:16.036731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23003181-00d1-4369-bb29-6fd3a3a786ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:16.039429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-23003181-00d1-4369-bb29-6fd3a3a786ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:16.042039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 890/1772] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:16.043564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2808a-8c9c-482f-b57c-bc428067c8bc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:16.043931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2808a-8c9c-482f-b57c-bc428067c8bc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:16.044214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2808a-8c9c-482f-b57c-bc428067c8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:16.062766 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23003181-00d1-4369-bb29-6fd3a3a786ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:16.065898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23003181-00d1-4369-bb29-6fd3a3a786ee service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:16.109981 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f0c2808a-8c9c-482f-b57c-bc428067c8bc None None] MFA Rules not processed for user `6bb5a1f18dff4395bec9c205a9937493`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:16.111506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 883/1773] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:16.118051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f0c2808a-8c9c-482f-b57c-bc428067c8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:16.119531 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23003181-00d1-4369-bb29-6fd3a3a786ee service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:16.121813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-23003181-00d1-4369-bb29-6fd3a3a786ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:16.165624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23003181-00d1-4369-bb29-6fd3a3a786ee service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:16.179178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:16.180093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:16.180373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:16.182865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] PATH_INFO: `/v3/projects/8057f6b1fb7b4efa883c29501728893e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:16.184625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8057f6b1fb7b4efa883c29501728893e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:16.190137 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:16.224044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:16.268944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 891/1774] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:16.270134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2808a-8c9c-482f-b57c-bc428067c8bc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:16.273308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f0c2808a-8c9c-482f-b57c-bc428067c8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:16.302896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8057f6b1fb7b4efa883c29501728893e', 'request_id': 'req-463f69c5-e0a2-454f-a640-f28d4fc32ccf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:16.309321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2808a-8c9c-482f-b57c-bc428067c8bc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:16.317154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8057f6b1fb7b4efa883c29501728893e', 'request_id': 'req-463f69c5-e0a2-454f-a640-f28d4fc32ccf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:16.331810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8057f6b1fb7b4efa883c29501728893e', 'request_id': 'req-463f69c5-e0a2-454f-a640-f28d4fc32ccf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:16.364967 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] The token cache is being invalidate because project 8057f6b1fb7b4efa883c29501728893e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:16.365290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:16.367646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:16.372492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 884/1775] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:16 2022] DELETE /identity/v3/projects/8057f6b1fb7b4efa883c29501728893e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:16.373371 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:16.380253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:16.456838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d779de-9702-48e9-8754-a6a8a2dbd9e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:16.457660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d779de-9702-48e9-8754-a6a8a2dbd9e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:16.458672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d779de-9702-48e9-8754-a6a8a2dbd9e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:16.459066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d779de-9702-48e9-8754-a6a8a2dbd9e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:16.462913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d7d779de-9702-48e9-8754-a6a8a2dbd9e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:16.489289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d779de-9702-48e9-8754-a6a8a2dbd9e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:16.493557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d779de-9702-48e9-8754-a6a8a2dbd9e3 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:16.517957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 892/1776] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:16.520766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6de6d-8688-4b12-8150-5396f03b8222 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:16.520766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6de6d-8688-4b12-8150-5396f03b8222 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:16.521384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6de6d-8688-4b12-8150-5396f03b8222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:16.572813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1ee6de6d-8688-4b12-8150-5396f03b8222 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:16.572813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1ee6de6d-8688-4b12-8150-5396f03b8222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:16.573800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f69c5-e0a2-454f-a640-f28d4fc32ccf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:16.594245 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 893/1777] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:16.596497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada7ecc-3d3e-4f19-96cc-5a9c66860768 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:16.596790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada7ecc-3d3e-4f19-96cc-5a9c66860768 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:16.597203 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada7ecc-3d3e-4f19-96cc-5a9c66860768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:16.597619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b99c8-8c17-4e5c-98fb-a9f5655620dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:16.598312 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b99c8-8c17-4e5c-98fb-a9f5655620dc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:16.602711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b99c8-8c17-4e5c-98fb-a9f5655620dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:16.603986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b99c8-8c17-4e5c-98fb-a9f5655620dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:16.604797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b99c8-8c17-4e5c-98fb-a9f5655620dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:16.609076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b99c8-8c17-4e5c-98fb-a9f5655620dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:16.610803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b99c8-8c17-4e5c-98fb-a9f5655620dc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:16.636282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7ada7ecc-3d3e-4f19-96cc-5a9c66860768 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:16.640465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7ada7ecc-3d3e-4f19-96cc-5a9c66860768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:16.658772 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 885/1778] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:16.661208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b99c8-8c17-4e5c-98fb-a9f5655620dc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:16.666812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 894/1779] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:16.668001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada7ecc-3d3e-4f19-96cc-5a9c66860768 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:16.690171 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada7ecc-3d3e-4f19-96cc-5a9c66860768 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:16.694461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b99c8-8c17-4e5c-98fb-a9f5655620dc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:16.703583 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a307c6-bddb-4155-a897-336f3faa6451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:16.704290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a307c6-bddb-4155-a897-336f3faa6451 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:16.704573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a307c6-bddb-4155-a897-336f3faa6451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:16.704864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a307c6-bddb-4155-a897-336f3faa6451 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:16.705647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a307c6-bddb-4155-a897-336f3faa6451 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:16.705959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a307c6-bddb-4155-a897-336f3faa6451 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:16.707586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a307c6-bddb-4155-a897-336f3faa6451 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:16.714188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7968a-25c5-47e4-8d28-14df6bb27dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:16.722972 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7968a-25c5-47e4-8d28-14df6bb27dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:16.723317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7968a-25c5-47e4-8d28-14df6bb27dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:16.728874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7968a-25c5-47e4-8d28-14df6bb27dc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:16.731239 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7968a-25c5-47e4-8d28-14df6bb27dc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:16.737529 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7968a-25c5-47e4-8d28-14df6bb27dc1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:16.759793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 895/1780] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:02:16.765628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a307c6-bddb-4155-a897-336f3faa6451 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:16.783648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a307c6-bddb-4155-a897-336f3faa6451 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:16.791094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3c53e-f1c1-4e49-bc47-eeecdf62330a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:16.792103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3c53e-f1c1-4e49-bc47-eeecdf62330a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:16.792538 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3c53e-f1c1-4e49-bc47-eeecdf62330a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:16.792987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3c53e-f1c1-4e49-bc47-eeecdf62330a admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:16.793843 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3c53e-f1c1-4e49-bc47-eeecdf62330a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:16.796517 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3c53e-f1c1-4e49-bc47-eeecdf62330a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:16.828026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 886/1781] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:16 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:16.832920 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7968a-25c5-47e4-8d28-14df6bb27dc1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:16.875000 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 896/1782] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:16 2022] POST /identity/v3/projects => generated 369 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:16.876288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3c53e-f1c1-4e49-bc47-eeecdf62330a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:16.899353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:16.965650 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7968a-25c5-47e4-8d28-14df6bb27dc1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:17.031179 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3c53e-f1c1-4e49-bc47-eeecdf62330a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:17.044444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d76b91-53fa-481f-9e37-f564c21c1811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:17.045137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d76b91-53fa-481f-9e37-f564c21c1811 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:17.045405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d76b91-53fa-481f-9e37-f564c21c1811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:17.045682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d76b91-53fa-481f-9e37-f564c21c1811 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:17.046389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d76b91-53fa-481f-9e37-f564c21c1811 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:17.048363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d76b91-53fa-481f-9e37-f564c21c1811 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:17.103898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 897/1783] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:16 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:17.103898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d76b91-53fa-481f-9e37-f564c21c1811 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:17.104071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a9d76b91-53fa-481f-9e37-f564c21c1811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:17.119446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d76b91-53fa-481f-9e37-f564c21c1811 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:17.145873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:17.158617 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:17.160446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:17.160832 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] PATH_INFO: `/v3/projects/f12f6358915b44ce8f2891e509214d50` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:17.164494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f12f6358915b44ce8f2891e509214d50)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:17.168302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:17.172308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd99713-cbda-42fd-a19e-db68f44d6f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:17.175505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd99713-cbda-42fd-a19e-db68f44d6f72 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:17.176274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd99713-cbda-42fd-a19e-db68f44d6f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:17.176585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd99713-cbda-42fd-a19e-db68f44d6f72 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:17.180342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd99713-cbda-42fd-a19e-db68f44d6f72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:17.184418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd99713-cbda-42fd-a19e-db68f44d6f72 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:17.255577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 887/1784] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:16 2022] POST /identity/v3/users => generated 287 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:17.255953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:17.257478 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd99713-cbda-42fd-a19e-db68f44d6f72 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:17.296727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd99713-cbda-42fd-a19e-db68f44d6f72 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:17.313114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f12f6358915b44ce8f2891e509214d50', 'request_id': 'req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:17.332310 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f12f6358915b44ce8f2891e509214d50', 'request_id': 'req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:17.342445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f12f6358915b44ce8f2891e509214d50', 'request_id': 'req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:17.367753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] The token cache is being invalidate because project f12f6358915b44ce8f2891e509214d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:17.368069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:17.369369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:17.377608 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 898/1785] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:17 2022] DELETE /identity/v3/projects/f12f6358915b44ce8f2891e509214d50 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:17.378848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:17.383077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:17.430776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b12676-f958-4bcf-93a1-e18dbfa04d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:17.432441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b12676-f958-4bcf-93a1-e18dbfa04d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:17.432904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b12676-f958-4bcf-93a1-e18dbfa04d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:17.433469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b12676-f958-4bcf-93a1-e18dbfa04d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:17.435307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b12676-f958-4bcf-93a1-e18dbfa04d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:17.437272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b12676-f958-4bcf-93a1-e18dbfa04d42 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:17.460907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 888/1786] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:17.461853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b12676-f958-4bcf-93a1-e18dbfa04d42 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:17.464742 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f6b12676-f958-4bcf-93a1-e18dbfa04d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:17.490017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b12676-f958-4bcf-93a1-e18dbfa04d42 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:17.499053 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04788bac-84a4-418b-9a3a-508c3cd75f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:17.499816 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788bac-84a4-418b-9a3a-508c3cd75f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:17.500109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788bac-84a4-418b-9a3a-508c3cd75f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:17.500398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788bac-84a4-418b-9a3a-508c3cd75f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:17.501725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04788bac-84a4-418b-9a3a-508c3cd75f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:17.503834 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04788bac-84a4-418b-9a3a-508c3cd75f63 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:17.528804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 889/1787] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:17.531213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb6755-c6f0-4e2e-9bad-a6da61982480 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:17.531918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb6755-c6f0-4e2e-9bad-a6da61982480 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:17.532357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb6755-c6f0-4e2e-9bad-a6da61982480 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:17.534242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 890/1788] 10.222.0.65 () {54 vars in 960 bytes} [Thu Jun 23 11:02:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:17.535532 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a059f-e8c8-496c-a96e-72347ff21276 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:17.536183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a059f-e8c8-496c-a96e-72347ff21276 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:17.536622 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a059f-e8c8-496c-a96e-72347ff21276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:17.568122 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-355a059f-e8c8-496c-a96e-72347ff21276 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:17.569301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-355a059f-e8c8-496c-a96e-72347ff21276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:17.577627 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4db71-ac96-4f5c-8951-a40e4fa38bdc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:17.582036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 891/1789] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:17.583040 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a059f-e8c8-496c-a96e-72347ff21276 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:17.589930 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-355a059f-e8c8-496c-a96e-72347ff21276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:17.592225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c9e9e-a99f-4489-8457-b1a2313b0460 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:17.593025 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c9e9e-a99f-4489-8457-b1a2313b0460 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:17.593276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c9e9e-a99f-4489-8457-b1a2313b0460 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:17.593703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c9e9e-a99f-4489-8457-b1a2313b0460 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:17.595494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6e1c9e9e-a99f-4489-8457-b1a2313b0460 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:17.626125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a059f-e8c8-496c-a96e-72347ff21276 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:17.636418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45968e8b-fea7-4e7f-a35f-792363d8a6f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:17.637527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45968e8b-fea7-4e7f-a35f-792363d8a6f3 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:17.638159 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45968e8b-fea7-4e7f-a35f-792363d8a6f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:17.640276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45968e8b-fea7-4e7f-a35f-792363d8a6f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:17.642591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-45968e8b-fea7-4e7f-a35f-792363d8a6f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:17.697559 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c9e9e-a99f-4489-8457-b1a2313b0460 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:17.700174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c9e9e-a99f-4489-8457-b1a2313b0460 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:17.716399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 899/1790] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:17.717105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:17.733790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c9e9e-a99f-4489-8457-b1a2313b0460 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:17.737246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6e1c9e9e-a99f-4489-8457-b1a2313b0460 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:17.762312 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45968e8b-fea7-4e7f-a35f-792363d8a6f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:17.765841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45968e8b-fea7-4e7f-a35f-792363d8a6f3 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:17.768450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c9e9e-a99f-4489-8457-b1a2313b0460 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:17.781506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291576fa-ae82-49d0-a752-f30cfbfd7047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:17.782210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291576fa-ae82-49d0-a752-f30cfbfd7047 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:17.784906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291576fa-ae82-49d0-a752-f30cfbfd7047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:17.785277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291576fa-ae82-49d0-a752-f30cfbfd7047 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:17.786053 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291576fa-ae82-49d0-a752-f30cfbfd7047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:17.789515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291576fa-ae82-49d0-a752-f30cfbfd7047 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:17.790409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 892/1791] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:17.791703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45968e8b-fea7-4e7f-a35f-792363d8a6f3 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:17.806397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45968e8b-fea7-4e7f-a35f-792363d8a6f3 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:17.816448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3be508-ed62-49ac-893c-03a383dd4d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:17.817357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3be508-ed62-49ac-893c-03a383dd4d0b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:17.817685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3be508-ed62-49ac-893c-03a383dd4d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:17.819144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3be508-ed62-49ac-893c-03a383dd4d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:17.819304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 900/1792] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:17.820400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3be508-ed62-49ac-893c-03a383dd4d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:17.822694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3be508-ed62-49ac-893c-03a383dd4d0b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:17.825467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291576fa-ae82-49d0-a752-f30cfbfd7047 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:17.829374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-291576fa-ae82-49d0-a752-f30cfbfd7047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:17.874127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 893/1793] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:17.876731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ff2f-2e0b-4f79-8183-1ef83b15f617 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:17.882033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ff2f-2e0b-4f79-8183-1ef83b15f617 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:17.884434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ff2f-2e0b-4f79-8183-1ef83b15f617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:17.951363 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e851ff2f-2e0b-4f79-8183-1ef83b15f617 None None] MFA Rules not processed for user `18fc0e7956b14a49beab025922e8ff81`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:17.954911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e851ff2f-2e0b-4f79-8183-1ef83b15f617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:17.993566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:18.066191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291576fa-ae82-49d0-a752-f30cfbfd7047 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:18.079077 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b339d-b298-49ea-a5ee-0d0988d7a921 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:18.080004 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b339d-b298-49ea-a5ee-0d0988d7a921 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:18.081222 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b339d-b298-49ea-a5ee-0d0988d7a921 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:18.082196 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b339d-b298-49ea-a5ee-0d0988d7a921 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:18.100328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-402b339d-b298-49ea-a5ee-0d0988d7a921 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:18.128285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b339d-b298-49ea-a5ee-0d0988d7a921 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:18.129828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b339d-b298-49ea-a5ee-0d0988d7a921 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:18.136642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 894/1794] 10.222.0.65 () {58 vars in 1057 bytes} [Thu Jun 23 11:02:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:18.138633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa607a3-be04-4e84-827c-2357f12eabc0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:18.139081 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa607a3-be04-4e84-827c-2357f12eabc0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:18.139364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa607a3-be04-4e84-827c-2357f12eabc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:18.171984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5fa607a3-be04-4e84-827c-2357f12eabc0 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:18.172162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 901/1795] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:18.173165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b339d-b298-49ea-a5ee-0d0988d7a921 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:18.174167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5fa607a3-be04-4e84-827c-2357f12eabc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:18.184061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 895/1796] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:18.185154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa607a3-be04-4e84-827c-2357f12eabc0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:18.195616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b339d-b298-49ea-a5ee-0d0988d7a921 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:18.200438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa607a3-be04-4e84-827c-2357f12eabc0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:18.220607 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311aca6-64d2-4c7f-9557-fa1cf4b80159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:18.221991 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311aca6-64d2-4c7f-9557-fa1cf4b80159 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:18.223887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9208a0-3e35-4b78-b6fd-d5f18147d8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:18.224869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9208a0-3e35-4b78-b6fd-d5f18147d8d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:18.226340 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311aca6-64d2-4c7f-9557-fa1cf4b80159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:18.226964 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311aca6-64d2-4c7f-9557-fa1cf4b80159 admin admin] PATH_INFO: `/v3/projects/8cbefe5f6d304d63be1bbd098405866b/users/c3138a6c6cf94431b916ab30ba3eda81/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:18.228150 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9208a0-3e35-4b78-b6fd-d5f18147d8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:18.228661 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9208a0-3e35-4b78-b6fd-d5f18147d8d2 admin admin] PATH_INFO: `/v3/projects/8ade814d33ff4327b2d47b4f5f8ab168/users/6bc813ffe96a45f4b49087c4fe12e823/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:18.266883 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311aca6-64d2-4c7f-9557-fa1cf4b80159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbefe5f6d304d63be1bbd098405866b, user_id=c3138a6c6cf94431b916ab30ba3eda81, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:18.270034 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311aca6-64d2-4c7f-9557-fa1cf4b80159 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:18.275973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9208a0-3e35-4b78-b6fd-d5f18147d8d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ade814d33ff4327b2d47b4f5f8ab168, user_id=6bc813ffe96a45f4b49087c4fe12e823, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:18.279348 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9208a0-3e35-4b78-b6fd-d5f18147d8d2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:18.314380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 896/1797] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:18 2022] PUT /identity/v3/projects/8ade814d33ff4327b2d47b4f5f8ab168/users/6bc813ffe96a45f4b49087c4fe12e823/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:18.316927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 902/1798] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:18 2022] PUT /identity/v3/projects/8cbefe5f6d304d63be1bbd098405866b/users/c3138a6c6cf94431b916ab30ba3eda81/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:18.320426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9208a0-3e35-4b78-b6fd-d5f18147d8d2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:18.321014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311aca6-64d2-4c7f-9557-fa1cf4b80159 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:18.491591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311aca6-64d2-4c7f-9557-fa1cf4b80159 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:18.514227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9208a0-3e35-4b78-b6fd-d5f18147d8d2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:18.527591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03fa3e-a72f-4c37-bdc6-b10f4fde3043 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:18.528304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03fa3e-a72f-4c37-bdc6-b10f4fde3043 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:18.528561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03fa3e-a72f-4c37-bdc6-b10f4fde3043 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:18.528839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03fa3e-a72f-4c37-bdc6-b10f4fde3043 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:18.560902 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03fa3e-a72f-4c37-bdc6-b10f4fde3043 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:18.566770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03fa3e-a72f-4c37-bdc6-b10f4fde3043 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:18.598352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 897/1799] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:18.600237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03fa3e-a72f-4c37-bdc6-b10f4fde3043 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:18.605067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7a03fa3e-a72f-4c37-bdc6-b10f4fde3043 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:18.623486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93244428-6942-40d2-af3d-13748c7f6409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:18.624221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93244428-6942-40d2-af3d-13748c7f6409 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:18.624505 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93244428-6942-40d2-af3d-13748c7f6409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:18.624781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93244428-6942-40d2-af3d-13748c7f6409 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:18.625526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93244428-6942-40d2-af3d-13748c7f6409 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:18.626390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93244428-6942-40d2-af3d-13748c7f6409 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:18.627795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93244428-6942-40d2-af3d-13748c7f6409 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:18.640180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03fa3e-a72f-4c37-bdc6-b10f4fde3043 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:18.651795 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:18.655827 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:18.658702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:18.659011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] PATH_INFO: `/v3/users/73a1bf83e2f640328ed4fefe3dc0018b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:18.662465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 903/1800] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:18.663447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a1bf83e2f640328ed4fefe3dc0018b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:18.665372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab5c9b-39fd-43e5-b61b-5604e3d5856e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:18.665509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab5c9b-39fd-43e5-b61b-5604e3d5856e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:18.665783 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab5c9b-39fd-43e5-b61b-5604e3d5856e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:18.673551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:18.721055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-18ab5c9b-39fd-43e5-b61b-5604e3d5856e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:18.722524 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-18ab5c9b-39fd-43e5-b61b-5604e3d5856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:18.735621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:18.748468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 904/1801] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:18.754923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cbb4d-32ee-4c7f-94a0-9963bf48086b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:18.757420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cbb4d-32ee-4c7f-94a0-9963bf48086b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:18.757935 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cbb4d-32ee-4c7f-94a0-9963bf48086b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:18.794632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a1bf83e2f640328ed4fefe3dc0018b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:18.795406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a1bf83e2f640328ed4fefe3dc0018b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:18.815117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a1bf83e2f640328ed4fefe3dc0018b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:18.835799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a77cbb4d-32ee-4c7f-94a0-9963bf48086b None None] MFA Rules not processed for user `c3138a6c6cf94431b916ab30ba3eda81`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:18.845287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a77cbb4d-32ee-4c7f-94a0-9963bf48086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:18.887305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a1bf83e2f640328ed4fefe3dc0018b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:18.912955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a1bf83e2f640328ed4fefe3dc0018b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:18.921158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:18.924804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 898/1802] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:18 2022] DELETE /identity/v3/users/73a1bf83e2f640328ed4fefe3dc0018b => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:18.927152 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:18.930538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:19.022994 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 905/1803] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:19.024239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cbb4d-32ee-4c7f-94a0-9963bf48086b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:19.028364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a77cbb4d-32ee-4c7f-94a0-9963bf48086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:19.158602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecd4e2-6cad-4873-8ad7-9d97fcb5ed7a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:19.174966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9135a-4302-4666-895a-f7a37433b129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:19.175733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9135a-4302-4666-895a-f7a37433b129 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:19.176028 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9135a-4302-4666-895a-f7a37433b129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:19.176343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9135a-4302-4666-895a-f7a37433b129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:19.177163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e9135a-4302-4666-895a-f7a37433b129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:19.186552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e9135a-4302-4666-895a-f7a37433b129 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:19.226072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cbb4d-32ee-4c7f-94a0-9963bf48086b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:19.338053 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 899/1804] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:18 2022] POST /identity/v3/projects => generated 401 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:19.340262 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193811f0-d7f4-4f88-be7e-10ed18a2a7ef None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:19.340730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193811f0-d7f4-4f88-be7e-10ed18a2a7ef None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:19.341211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193811f0-d7f4-4f88-be7e-10ed18a2a7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:19.383807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-193811f0-d7f4-4f88-be7e-10ed18a2a7ef None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:19.385066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-193811f0-d7f4-4f88-be7e-10ed18a2a7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:19.432308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20645f-3269-4af2-b255-da749d3f4168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:19.434247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20645f-3269-4af2-b255-da749d3f4168 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:19.438067 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20645f-3269-4af2-b255-da749d3f4168 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:19.438383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20645f-3269-4af2-b255-da749d3f4168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:19.447602 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fa20645f-3269-4af2-b255-da749d3f4168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:19.487648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20645f-3269-4af2-b255-da749d3f4168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:19.492322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20645f-3269-4af2-b255-da749d3f4168 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:19.513042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 900/1805] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:19.513917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193811f0-d7f4-4f88-be7e-10ed18a2a7ef None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:19.526804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-193811f0-d7f4-4f88-be7e-10ed18a2a7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:19.529564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 906/1806] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:19.532245 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20645f-3269-4af2-b255-da749d3f4168 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:19.557378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20645f-3269-4af2-b255-da749d3f4168 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:19.558044 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193811f0-d7f4-4f88-be7e-10ed18a2a7ef None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:19.568496 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8686838-dfc8-460a-86ef-a5e6b253d18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:19.568740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58963d0-4289-482c-864a-5eb4b04c4372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:19.569803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58963d0-4289-482c-864a-5eb4b04c4372 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:19.570531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8686838-dfc8-460a-86ef-a5e6b253d18a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:19.570958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8686838-dfc8-460a-86ef-a5e6b253d18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:19.571423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58963d0-4289-482c-864a-5eb4b04c4372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:19.571862 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8686838-dfc8-460a-86ef-a5e6b253d18a admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:19.573497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58963d0-4289-482c-864a-5eb4b04c4372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:19.575007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58963d0-4289-482c-864a-5eb4b04c4372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:19.575470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8686838-dfc8-460a-86ef-a5e6b253d18a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:19.579570 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8686838-dfc8-460a-86ef-a5e6b253d18a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:19.580861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58963d0-4289-482c-864a-5eb4b04c4372 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:19.632513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 907/1807] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:19 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:19.634969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8686838-dfc8-460a-86ef-a5e6b253d18a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:19.668572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e8686838-dfc8-460a-86ef-a5e6b253d18a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:19.673539 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 901/1808] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:19 2022] POST /identity/v3/projects => generated 369 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:19.673539 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58963d0-4289-482c-864a-5eb4b04c4372 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:19.685522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:19.831347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8686838-dfc8-460a-86ef-a5e6b253d18a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:19.841699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58963d0-4289-482c-864a-5eb4b04c4372 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:19.855988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9e55d-e649-4a2d-a074-c44e772dae33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:19.858032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9e55d-e649-4a2d-a074-c44e772dae33 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:19.858429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9e55d-e649-4a2d-a074-c44e772dae33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:19.860014 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9e55d-e649-4a2d-a074-c44e772dae33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:19.861537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9e55d-e649-4a2d-a074-c44e772dae33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:19.864102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9e55d-e649-4a2d-a074-c44e772dae33 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:19.905376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 902/1809] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:19.905376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9e55d-e649-4a2d-a074-c44e772dae33 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:19.906654 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:19.939997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9e55d-e649-4a2d-a074-c44e772dae33 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:19.957034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd687f52-695f-4f53-924f-43a436281489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:19.957707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd687f52-695f-4f53-924f-43a436281489 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:19.957993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd687f52-695f-4f53-924f-43a436281489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:19.958269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd687f52-695f-4f53-924f-43a436281489 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:19.958976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd687f52-695f-4f53-924f-43a436281489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:19.964500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd687f52-695f-4f53-924f-43a436281489 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:19.983837 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3db1a3-fe11-48bb-92a6-0df82650a848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:19.984389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3db1a3-fe11-48bb-92a6-0df82650a848 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:19.984670 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3db1a3-fe11-48bb-92a6-0df82650a848 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:19.984947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3db1a3-fe11-48bb-92a6-0df82650a848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:20.025243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3db1a3-fe11-48bb-92a6-0df82650a848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:20.028564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3db1a3-fe11-48bb-92a6-0df82650a848 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:20.029077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 903/1810] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:19 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:20.029910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd687f52-695f-4f53-924f-43a436281489 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:20.058726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd687f52-695f-4f53-924f-43a436281489 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:20.060795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 908/1811] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:20.070055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3988ac-caa3-4c2a-8d10-dd75104eeaa5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:20.070438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3988ac-caa3-4c2a-8d10-dd75104eeaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:20.070722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3988ac-caa3-4c2a-8d10-dd75104eeaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:20.077180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4ea7a-383b-482d-9114-27ba0008aef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:20.077878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4ea7a-383b-482d-9114-27ba0008aef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:20.078162 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4ea7a-383b-482d-9114-27ba0008aef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:20.078441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4ea7a-383b-482d-9114-27ba0008aef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:20.079190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4ea7a-383b-482d-9114-27ba0008aef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:20.080833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4ea7a-383b-482d-9114-27ba0008aef1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:20.130757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 904/1812] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:20.131638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4ea7a-383b-482d-9114-27ba0008aef1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:20.136128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-da3988ac-caa3-4c2a-8d10-dd75104eeaa5 None None] MFA Rules not processed for user `6bb5a1f18dff4395bec9c205a9937493`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:20.140972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-da3988ac-caa3-4c2a-8d10-dd75104eeaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:20.177119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4ea7a-383b-482d-9114-27ba0008aef1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:20.200213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffade0-2714-4d19-90c8-c14f1f62a24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:20.204381 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffade0-2714-4d19-90c8-c14f1f62a24c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:20.207389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffade0-2714-4d19-90c8-c14f1f62a24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:20.207751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffade0-2714-4d19-90c8-c14f1f62a24c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:20.215037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ffade0-2714-4d19-90c8-c14f1f62a24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:20.216921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ffade0-2714-4d19-90c8-c14f1f62a24c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:20.276184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 905/1813] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:20.277089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffade0-2714-4d19-90c8-c14f1f62a24c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:20.322774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffade0-2714-4d19-90c8-c14f1f62a24c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:20.357851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a74ea-05b6-4fed-8eb3-8dcdf987b15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:20.368660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a74ea-05b6-4fed-8eb3-8dcdf987b15e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:20.371910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a74ea-05b6-4fed-8eb3-8dcdf987b15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:20.372203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a74ea-05b6-4fed-8eb3-8dcdf987b15e admin admin] PATH_INFO: `/v3/projects/5f9695f5a86744e39f11b2fc4b9b20df/users/426cc6ea65fb4f90b3fd69bd90cd4287/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:20.396224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 909/1814] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:20.407560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3988ac-caa3-4c2a-8d10-dd75104eeaa5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:20.436387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a74ea-05b6-4fed-8eb3-8dcdf987b15e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9695f5a86744e39f11b2fc4b9b20df, user_id=426cc6ea65fb4f90b3fd69bd90cd4287, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:20.439090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a74ea-05b6-4fed-8eb3-8dcdf987b15e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:20.444115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3988ac-caa3-4c2a-8d10-dd75104eeaa5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:20.457952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84d629-97c7-423b-ad89-6e59fed2a7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:20.458871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84d629-97c7-423b-ad89-6e59fed2a7e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:20.459149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84d629-97c7-423b-ad89-6e59fed2a7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:20.459442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84d629-97c7-423b-ad89-6e59fed2a7e1 admin admin] PATH_INFO: `/v3/projects/5890071aeae048548df3f156158d4981/users/0f3ab50298874bc5a3e52cbde471e335/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:20.478636 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 906/1815] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:20 2022] PUT /identity/v3/projects/5f9695f5a86744e39f11b2fc4b9b20df/users/426cc6ea65fb4f90b3fd69bd90cd4287/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:20.490036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a29914-34fb-48a0-9e0d-8dbfe1379e50 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:20.490368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a29914-34fb-48a0-9e0d-8dbfe1379e50 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:20.491197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a29914-34fb-48a0-9e0d-8dbfe1379e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:20.513472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84d629-97c7-423b-ad89-6e59fed2a7e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5890071aeae048548df3f156158d4981, user_id=0f3ab50298874bc5a3e52cbde471e335, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:20.525184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84d629-97c7-423b-ad89-6e59fed2a7e1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:20.570603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 910/1816] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:20 2022] PUT /identity/v3/projects/5890071aeae048548df3f156158d4981/users/0f3ab50298874bc5a3e52cbde471e335/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:20.576757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84d629-97c7-423b-ad89-6e59fed2a7e1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:20.597981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d3a29914-34fb-48a0-9e0d-8dbfe1379e50 None None] MFA Rules not processed for user `426cc6ea65fb4f90b3fd69bd90cd4287`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:20.605166 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d3a29914-34fb-48a0-9e0d-8dbfe1379e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:20.812605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84d629-97c7-423b-ad89-6e59fed2a7e1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:20.832688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d26f2-3c16-48fe-846c-cac66d65ea90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:20.833295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d26f2-3c16-48fe-846c-cac66d65ea90 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:20.833585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d26f2-3c16-48fe-846c-cac66d65ea90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:20.833864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d26f2-3c16-48fe-846c-cac66d65ea90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:20.834617 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d26f2-3c16-48fe-846c-cac66d65ea90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:20.843022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d26f2-3c16-48fe-846c-cac66d65ea90 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:20.854161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 907/1817] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:20.855138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a29914-34fb-48a0-9e0d-8dbfe1379e50 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:20.883122 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 911/1818] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:20.887579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d26f2-3c16-48fe-846c-cac66d65ea90 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:20.894062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a29914-34fb-48a0-9e0d-8dbfe1379e50 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:20.911540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d981c1-bcda-48d8-a11f-d818bbbe7d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:20.915253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d981c1-bcda-48d8-a11f-d818bbbe7d35 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:20.915518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d981c1-bcda-48d8-a11f-d818bbbe7d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:20.915835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d981c1-bcda-48d8-a11f-d818bbbe7d35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:20.917975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d981c1-bcda-48d8-a11f-d818bbbe7d35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:20.920681 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d26f2-3c16-48fe-846c-cac66d65ea90 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:20.927468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d981c1-bcda-48d8-a11f-d818bbbe7d35 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:20.939390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599afb7c-9a53-435b-9c36-d90f12034695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:20.940723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599afb7c-9a53-435b-9c36-d90f12034695 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:20.940961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599afb7c-9a53-435b-9c36-d90f12034695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:20.942549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599afb7c-9a53-435b-9c36-d90f12034695 admin admin] PATH_INFO: `/v3/projects/5890071aeae048548df3f156158d4981/users/0f3ab50298874bc5a3e52cbde471e335/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:20.950764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599afb7c-9a53-435b-9c36-d90f12034695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5890071aeae048548df3f156158d4981, user_id=0f3ab50298874bc5a3e52cbde471e335, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:20.958044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599afb7c-9a53-435b-9c36-d90f12034695 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:20.988988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 912/1819] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:20 2022] PUT /identity/v3/projects/5890071aeae048548df3f156158d4981/users/0f3ab50298874bc5a3e52cbde471e335/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:21.002341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b2c0a-b5e7-4097-adae-c06ba83941f1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:21.002626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b2c0a-b5e7-4097-adae-c06ba83941f1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:21.002913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b2c0a-b5e7-4097-adae-c06ba83941f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:21.030568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 908/1820] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:20 2022] POST /identity/v3/projects => generated 401 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:21.035006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d981c1-bcda-48d8-a11f-d818bbbe7d35 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:21.049462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:21.058882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9b8b2c0a-b5e7-4097-adae-c06ba83941f1 None None] MFA Rules not processed for user `0f3ab50298874bc5a3e52cbde471e335`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:21.061308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9b8b2c0a-b5e7-4097-adae-c06ba83941f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:21.122598 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:21.275676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 913/1821] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:21 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:21.277962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b2c0a-b5e7-4097-adae-c06ba83941f1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:21.288608 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9b8b2c0a-b5e7-4097-adae-c06ba83941f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:21.295583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d981c1-bcda-48d8-a11f-d818bbbe7d35 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:21.320713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b5390-dd2f-497e-b324-896970c5f552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:21.321611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b5390-dd2f-497e-b324-896970c5f552 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:21.322005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b5390-dd2f-497e-b324-896970c5f552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:21.323683 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b5390-dd2f-497e-b324-896970c5f552 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:21.324556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b5390-dd2f-497e-b324-896970c5f552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:21.326725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b5390-dd2f-497e-b324-896970c5f552 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:21.337613 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b2c0a-b5e7-4097-adae-c06ba83941f1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:21.351590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:21.355213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:21.355213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:21.356420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] PATH_INFO: `/v3/projects/7718f93df9bb4e1d8312a606d25d7bf3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:21.359645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7718f93df9bb4e1d8312a606d25d7bf3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:21.361716 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:21.376050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 909/1822] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:21 2022] POST /identity/v3/users => generated 307 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:21.380103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b5390-dd2f-497e-b324-896970c5f552 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:21.385986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d51b5390-dd2f-497e-b324-896970c5f552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:21.487529 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7718f93df9bb4e1d8312a606d25d7bf3', 'request_id': 'req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:21.499502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7718f93df9bb4e1d8312a606d25d7bf3', 'request_id': 'req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:21.508275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7718f93df9bb4e1d8312a606d25d7bf3', 'request_id': 'req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:21.534138 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] The token cache is being invalidate because project 7718f93df9bb4e1d8312a606d25d7bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:21.534666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:21.540034 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:21.540994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b5390-dd2f-497e-b324-896970c5f552 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:21.542828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 914/1823] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:21 2022] DELETE /identity/v3/projects/7718f93df9bb4e1d8312a606d25d7bf3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:21.543850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:21.547050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:21.665750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538d405-184c-4f73-8a80-a163f6784f45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:21.666521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538d405-184c-4f73-8a80-a163f6784f45 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:21.666916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538d405-184c-4f73-8a80-a163f6784f45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:21.667305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538d405-184c-4f73-8a80-a163f6784f45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:21.674531 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c538d405-184c-4f73-8a80-a163f6784f45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:21.756492 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c9efa-018c-4bc0-995c-2f5e942cb4a5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:21.772293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc347bf-f1aa-43a0-8970-b972f832877e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:21.775363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc347bf-f1aa-43a0-8970-b972f832877e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:21.776883 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc347bf-f1aa-43a0-8970-b972f832877e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:21.777333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc347bf-f1aa-43a0-8970-b972f832877e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:21.778700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc347bf-f1aa-43a0-8970-b972f832877e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:21.784578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc347bf-f1aa-43a0-8970-b972f832877e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:21.812494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 915/1824] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:21.813412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc347bf-f1aa-43a0-8970-b972f832877e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:21.818601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2cc347bf-f1aa-43a0-8970-b972f832877e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:21.822800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538d405-184c-4f73-8a80-a163f6784f45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:21.824233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538d405-184c-4f73-8a80-a163f6784f45 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:21.830065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:21.841591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 910/1825] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:21 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:21.842638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538d405-184c-4f73-8a80-a163f6784f45 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:21.858860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538d405-184c-4f73-8a80-a163f6784f45 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:21.866511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790faabe-c9f9-4d44-86dd-d606fd6a41d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:21.867876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790faabe-c9f9-4d44-86dd-d606fd6a41d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:21.868423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790faabe-c9f9-4d44-86dd-d606fd6a41d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:21.868929 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790faabe-c9f9-4d44-86dd-d606fd6a41d7 admin admin] PATH_INFO: `/v3/projects/2095f76e57644518a4de181ee522416d/users/739a7c8b5b414f0eaa061bb4e8c3871f/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:21.873130 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc347bf-f1aa-43a0-8970-b972f832877e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:21.893640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:21.894428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790faabe-c9f9-4d44-86dd-d606fd6a41d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2095f76e57644518a4de181ee522416d, user_id=739a7c8b5b414f0eaa061bb4e8c3871f, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:21.896181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:21.900774 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:21.901461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790faabe-c9f9-4d44-86dd-d606fd6a41d7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:21.903069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] PATH_INFO: `/v3/users/e10ceb6dd39a4cdd99adaab248dcdf8e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:21.909398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=e10ceb6dd39a4cdd99adaab248dcdf8e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:21.924745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:21.935878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 911/1826] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:21 2022] PUT /identity/v3/projects/2095f76e57644518a4de181ee522416d/users/739a7c8b5b414f0eaa061bb4e8c3871f/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:21.940215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790faabe-c9f9-4d44-86dd-d606fd6a41d7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:21.945408 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-790faabe-c9f9-4d44-86dd-d606fd6a41d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:21.994786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:22.015473 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e10ceb6dd39a4cdd99adaab248dcdf8e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:22.016651 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e10ceb6dd39a4cdd99adaab248dcdf8e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:22.017500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e10ceb6dd39a4cdd99adaab248dcdf8e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:22.049827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e10ceb6dd39a4cdd99adaab248dcdf8e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:22.064408 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e10ceb6dd39a4cdd99adaab248dcdf8e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:22.104155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 916/1827] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:21 2022] DELETE /identity/v3/users/e10ceb6dd39a4cdd99adaab248dcdf8e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:22.106495 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:22.110636 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790faabe-c9f9-4d44-86dd-d606fd6a41d7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:22.111383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:22.237065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c15ae-8f74-435c-8ad1-0b608d3f72eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:22.237676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c15ae-8f74-435c-8ad1-0b608d3f72eb service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:22.237983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c15ae-8f74-435c-8ad1-0b608d3f72eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:22.238250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c15ae-8f74-435c-8ad1-0b608d3f72eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:22.245353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3c0c15ae-8f74-435c-8ad1-0b608d3f72eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:22.332813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec189991-05fb-4551-a1b3-304d3f2922ed admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:22.351973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe9ec1-161f-4302-a583-fb9b6cfd55aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:22.352714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe9ec1-161f-4302-a583-fb9b6cfd55aa admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:22.353006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe9ec1-161f-4302-a583-fb9b6cfd55aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:22.353771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe9ec1-161f-4302-a583-fb9b6cfd55aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:22.354040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe9ec1-161f-4302-a583-fb9b6cfd55aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:22.362386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe9ec1-161f-4302-a583-fb9b6cfd55aa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:22.383959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 917/1828] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:22.389907 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe9ec1-161f-4302-a583-fb9b6cfd55aa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:22.399918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c15ae-8f74-435c-8ad1-0b608d3f72eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:22.406432 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c15ae-8f74-435c-8ad1-0b608d3f72eb service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:22.408426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe9ec1-161f-4302-a583-fb9b6cfd55aa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:22.422772 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce746443-654c-43dd-9a92-1751d90302cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:22.423713 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce746443-654c-43dd-9a92-1751d90302cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:22.424006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce746443-654c-43dd-9a92-1751d90302cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:22.424289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce746443-654c-43dd-9a92-1751d90302cb admin admin] PATH_INFO: `/v3/projects/2095f76e57644518a4de181ee522416d/users/739a7c8b5b414f0eaa061bb4e8c3871f/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:22.432653 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 912/1829] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:22.434877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce746443-654c-43dd-9a92-1751d90302cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2095f76e57644518a4de181ee522416d, user_id=739a7c8b5b414f0eaa061bb4e8c3871f, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:22.440274 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce746443-654c-43dd-9a92-1751d90302cb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:22.478054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 918/1830] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:22 2022] PUT /identity/v3/projects/2095f76e57644518a4de181ee522416d/users/739a7c8b5b414f0eaa061bb4e8c3871f/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:22.480146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bb7ec-4dff-4d15-8eae-8d702e3e6787 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:22.480442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bb7ec-4dff-4d15-8eae-8d702e3e6787 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:22.480725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bb7ec-4dff-4d15-8eae-8d702e3e6787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:22.531479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-162bb7ec-4dff-4d15-8eae-8d702e3e6787 None None] MFA Rules not processed for user `739a7c8b5b414f0eaa061bb4e8c3871f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:22.532886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-162bb7ec-4dff-4d15-8eae-8d702e3e6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:22.609638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:22.720531 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 913/1831] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:22 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:22.734458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce746443-654c-43dd-9a92-1751d90302cb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:22.737283 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ce746443-654c-43dd-9a92-1751d90302cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:22.980686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce746443-654c-43dd-9a92-1751d90302cb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:23.002881 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1eef6a-439e-46e1-bffb-145508393746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:23.003470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1eef6a-439e-46e1-bffb-145508393746 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:23.005823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1eef6a-439e-46e1-bffb-145508393746 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:23.006129 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1eef6a-439e-46e1-bffb-145508393746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:23.017878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f97c7-75be-4d7d-9d38-7fa5c19cd8c8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:23.020726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f97c7-75be-4d7d-9d38-7fa5c19cd8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:23.021516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f97c7-75be-4d7d-9d38-7fa5c19cd8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:23.048416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1eef6a-439e-46e1-bffb-145508393746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:23.050021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1eef6a-439e-46e1-bffb-145508393746 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:23.109098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0f1f97c7-75be-4d7d-9d38-7fa5c19cd8c8 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:23.123140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f1f97c7-75be-4d7d-9d38-7fa5c19cd8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:23.130207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 919/1832] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:22 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:23.289711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1eef6a-439e-46e1-bffb-145508393746 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:23.295432 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 914/1833] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:02:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:23.305274 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1eef6a-439e-46e1-bffb-145508393746 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:23.331892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc23509-6e8e-450c-8802-dfb2a7fc5283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:23.332725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc23509-6e8e-450c-8802-dfb2a7fc5283 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:23.333198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc23509-6e8e-450c-8802-dfb2a7fc5283 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:23.333667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc23509-6e8e-450c-8802-dfb2a7fc5283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:23.337479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5fc23509-6e8e-450c-8802-dfb2a7fc5283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:23.386040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:23.439554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc23509-6e8e-450c-8802-dfb2a7fc5283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:23.441404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc23509-6e8e-450c-8802-dfb2a7fc5283 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:23.464602 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 920/1834] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:23.591013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f97c7-75be-4d7d-9d38-7fa5c19cd8c8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:23.596672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f1f97c7-75be-4d7d-9d38-7fa5c19cd8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:23.982127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f97c7-75be-4d7d-9d38-7fa5c19cd8c8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:23.989479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fdb0d-9805-4790-b681-2377f3606cd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:23.990168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fdb0d-9805-4790-b681-2377f3606cd3 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:23.990733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fdb0d-9805-4790-b681-2377f3606cd3 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:23.991993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fdb0d-9805-4790-b681-2377f3606cd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:24.043296 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fdb0d-9805-4790-b681-2377f3606cd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:24.046122 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fdb0d-9805-4790-b681-2377f3606cd3 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:24.104440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 915/1835] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:02:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:02:25.676757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc23509-6e8e-450c-8802-dfb2a7fc5283 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:25.678614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5fc23509-6e8e-450c-8802-dfb2a7fc5283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:25.717552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef3a75-8feb-4cf0-9bf0-ed24d61a2307 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:25.718133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef3a75-8feb-4cf0-9bf0-ed24d61a2307 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:25.718683 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef3a75-8feb-4cf0-9bf0-ed24d61a2307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:25.852487 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0fef3a75-8feb-4cf0-9bf0-ed24d61a2307 None None] MFA Rules not processed for user `6bc813ffe96a45f4b49087c4fe12e823`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:25.863047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0fef3a75-8feb-4cf0-9bf0-ed24d61a2307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:25.868460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc23509-6e8e-450c-8802-dfb2a7fc5283 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:25.886918 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:25.888075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:25.888361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:25.895690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] PATH_INFO: `/v3/projects/eb530f6181524731875b76a146ab1d45` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:25.895690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb530f6181524731875b76a146ab1d45)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:25.897152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:26.119199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb530f6181524731875b76a146ab1d45', 'request_id': 'req-33e2b185-4b8b-439b-b11e-7810d54fc01b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:26.138188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb530f6181524731875b76a146ab1d45', 'request_id': 'req-33e2b185-4b8b-439b-b11e-7810d54fc01b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:26.156399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb530f6181524731875b76a146ab1d45', 'request_id': 'req-33e2b185-4b8b-439b-b11e-7810d54fc01b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:26.168080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:26.199289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] The token cache is being invalidate because project eb530f6181524731875b76a146ab1d45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:26.199635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:26.203939 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 916/1836] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:26.206629 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a639b8b-02c6-44af-92af-43359de11a1d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:26.206779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a639b8b-02c6-44af-92af-43359de11a1d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:26.207033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:26.207355 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a639b8b-02c6-44af-92af-43359de11a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:26.219086 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 921/1837] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:25 2022] DELETE /identity/v3/projects/eb530f6181524731875b76a146ab1d45 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:26.220180 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:26.221742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:26.275346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9a639b8b-02c6-44af-92af-43359de11a1d None None] MFA Rules not processed for user `e96b136657864104a51900debd1fbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:26.280228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9a639b8b-02c6-44af-92af-43359de11a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:26.441909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2b185-4b8b-439b-b11e-7810d54fc01b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:26.466682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e0cf6-de98-4b33-a84b-f0dbffda2bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:26.467405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e0cf6-de98-4b33-a84b-f0dbffda2bbf admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:26.467740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e0cf6-de98-4b33-a84b-f0dbffda2bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:26.467974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e0cf6-de98-4b33-a84b-f0dbffda2bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:26.468878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e0cf6-de98-4b33-a84b-f0dbffda2bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:26.474539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e0cf6-de98-4b33-a84b-f0dbffda2bbf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:26.483922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 917/1838] 10.222.0.60 () {58 vars in 1015 bytes} [Thu Jun 23 11:02:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:26.502703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e457464-0772-49c7-a9cf-8058271dbad4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:26.503064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e457464-0772-49c7-a9cf-8058271dbad4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:26.503400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e457464-0772-49c7-a9cf-8058271dbad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:26.511132 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:26.566467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 922/1839] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:26 2022] POST /identity/v3/projects => generated 373 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:26.567943 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e0cf6-de98-4b33-a84b-f0dbffda2bbf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:26.570800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5e457464-0772-49c7-a9cf-8058271dbad4 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:26.573342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5e457464-0772-49c7-a9cf-8058271dbad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:26.590020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-085e0cf6-de98-4b33-a84b-f0dbffda2bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:26.700776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 918/1840] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:26.701876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e457464-0772-49c7-a9cf-8058271dbad4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:26.703444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5e457464-0772-49c7-a9cf-8058271dbad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:26.771113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e0cf6-de98-4b33-a84b-f0dbffda2bbf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:26.777642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e90ad2-bbca-4c17-bacb-be5af3203209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:26.778422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e90ad2-bbca-4c17-bacb-be5af3203209 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:26.778713 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e90ad2-bbca-4c17-bacb-be5af3203209 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:26.779200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e90ad2-bbca-4c17-bacb-be5af3203209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:26.822755 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e90ad2-bbca-4c17-bacb-be5af3203209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:26.824221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e90ad2-bbca-4c17-bacb-be5af3203209 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:26.868354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 923/1841] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:26.869414 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e90ad2-bbca-4c17-bacb-be5af3203209 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:26.901774 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e90ad2-bbca-4c17-bacb-be5af3203209 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:26.912546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9317c4-74b5-47d1-922d-52b237fcf7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:26.913873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9317c4-74b5-47d1-922d-52b237fcf7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:26.914175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9317c4-74b5-47d1-922d-52b237fcf7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:26.914480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9317c4-74b5-47d1-922d-52b237fcf7e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:26.915237 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9317c4-74b5-47d1-922d-52b237fcf7e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:26.917606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9317c4-74b5-47d1-922d-52b237fcf7e2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:26.938875 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e457464-0772-49c7-a9cf-8058271dbad4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:26.941779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b1e2a-e8ca-4700-8d17-534ea3581273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:26.942516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b1e2a-e8ca-4700-8d17-534ea3581273 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:26.942828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b1e2a-e8ca-4700-8d17-534ea3581273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:26.943110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b1e2a-e8ca-4700-8d17-534ea3581273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:26.955679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 924/1842] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:26 2022] POST /identity/v3/users => generated 293 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:26.957249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba529d4-01db-43d9-88ce-56f088f163dc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:26.957543 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba529d4-01db-43d9-88ce-56f088f163dc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:26.957827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba529d4-01db-43d9-88ce-56f088f163dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:26.960462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b1e2a-e8ca-4700-8d17-534ea3581273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:26.961932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b1e2a-e8ca-4700-8d17-534ea3581273 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:26.986002 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 919/1843] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:26.986320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b1e2a-e8ca-4700-8d17-534ea3581273 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:27.009508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b1e2a-e8ca-4700-8d17-534ea3581273 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:27.020533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3281b3b-e84d-4261-8247-73c5fb72bc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:27.021225 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3281b3b-e84d-4261-8247-73c5fb72bc59 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:27.021511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3281b3b-e84d-4261-8247-73c5fb72bc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:27.021788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3281b3b-e84d-4261-8247-73c5fb72bc59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:27.024523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3281b3b-e84d-4261-8247-73c5fb72bc59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:27.030318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3281b3b-e84d-4261-8247-73c5fb72bc59 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:27.034421 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cba529d4-01db-43d9-88ce-56f088f163dc None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:27.040434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cba529d4-01db-43d9-88ce-56f088f163dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:27.075588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 925/1844] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:27.075793 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 920/1845] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:27.076814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3281b3b-e84d-4261-8247-73c5fb72bc59 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:27.077702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b88dbd-a81a-46c9-a988-c18b53cb16bb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:27.078503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b88dbd-a81a-46c9-a988-c18b53cb16bb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:27.079068 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b88dbd-a81a-46c9-a988-c18b53cb16bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:27.098360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3281b3b-e84d-4261-8247-73c5fb72bc59 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:27.112304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b0b88dbd-a81a-46c9-a988-c18b53cb16bb None None] MFA Rules not processed for user `303e8aae1bd04228bedb145dec891a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:27.114964 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b0b88dbd-a81a-46c9-a988-c18b53cb16bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:27.117817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd2d26-e112-4aa5-8f46-127b7a576b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:27.124657 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd2d26-e112-4aa5-8f46-127b7a576b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:27.125132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd2d26-e112-4aa5-8f46-127b7a576b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:27.125757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd2d26-e112-4aa5-8f46-127b7a576b57 admin admin] PATH_INFO: `/v3/projects/0182c426a5a14ef7825c364740ead0ed/users/3abc9cd378424bb1a7d4b38cf21fd0fb/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:27.155000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd2d26-e112-4aa5-8f46-127b7a576b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0182c426a5a14ef7825c364740ead0ed, user_id=3abc9cd378424bb1a7d4b38cf21fd0fb, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:27.161970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd2d26-e112-4aa5-8f46-127b7a576b57 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:27.212380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 921/1846] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:27 2022] PUT /identity/v3/projects/0182c426a5a14ef7825c364740ead0ed/users/3abc9cd378424bb1a7d4b38cf21fd0fb/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:27.213325 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd2d26-e112-4aa5-8f46-127b7a576b57 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:27.262856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:27.343693 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 926/1847] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:27.344640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b88dbd-a81a-46c9-a988-c18b53cb16bb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:27.377714 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd2d26-e112-4aa5-8f46-127b7a576b57 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:27.383456 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997177e5-08c8-4d76-97d8-6854b0868e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:27.384171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997177e5-08c8-4d76-97d8-6854b0868e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:27.384594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997177e5-08c8-4d76-97d8-6854b0868e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:27.384881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997177e5-08c8-4d76-97d8-6854b0868e5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:27.385682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997177e5-08c8-4d76-97d8-6854b0868e5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:27.386002 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997177e5-08c8-4d76-97d8-6854b0868e5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:27.387886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997177e5-08c8-4d76-97d8-6854b0868e5b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:27.393711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b88dbd-a81a-46c9-a988-c18b53cb16bb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:27.406540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 922/1848] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:27.407450 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997177e5-08c8-4d76-97d8-6854b0868e5b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:27.411787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a99de-d116-4109-9922-49962e167a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:27.412483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a99de-d116-4109-9922-49962e167a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:27.412765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a99de-d116-4109-9922-49962e167a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:27.413081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a99de-d116-4109-9922-49962e167a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:27.413825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a99de-d116-4109-9922-49962e167a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:27.419915 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997177e5-08c8-4d76-97d8-6854b0868e5b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:27.427689 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55519ce3-24f3-4c21-a4e2-bd2deadb257c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:27.428434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55519ce3-24f3-4c21-a4e2-bd2deadb257c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:27.429280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55519ce3-24f3-4c21-a4e2-bd2deadb257c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:27.429563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55519ce3-24f3-4c21-a4e2-bd2deadb257c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:27.430444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55519ce3-24f3-4c21-a4e2-bd2deadb257c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:27.432010 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a99de-d116-4109-9922-49962e167a67 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:27.432842 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55519ce3-24f3-4c21-a4e2-bd2deadb257c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:27.475849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 927/1849] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:27.477503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a99de-d116-4109-9922-49962e167a67 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:27.489800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 923/1850] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:27 2022] POST /identity/v3/projects => generated 407 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:27.490865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55519ce3-24f3-4c21-a4e2-bd2deadb257c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:27.512664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a99de-d116-4109-9922-49962e167a67 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:27.691971 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55519ce3-24f3-4c21-a4e2-bd2deadb257c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:27.705451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80371e8f-1b2e-4c49-bb10-f41ea76d5b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:27.707430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80371e8f-1b2e-4c49-bb10-f41ea76d5b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:27.707752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80371e8f-1b2e-4c49-bb10-f41ea76d5b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:27.708042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80371e8f-1b2e-4c49-bb10-f41ea76d5b34 admin admin] PATH_INFO: `/v3/projects/0182c426a5a14ef7825c364740ead0ed/users/3abc9cd378424bb1a7d4b38cf21fd0fb/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:27.714676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80371e8f-1b2e-4c49-bb10-f41ea76d5b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0182c426a5a14ef7825c364740ead0ed, user_id=3abc9cd378424bb1a7d4b38cf21fd0fb, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:27.717394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cc09b-752d-4b17-9074-eb3c56037afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:27.718176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cc09b-752d-4b17-9074-eb3c56037afc admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:27.718360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cc09b-752d-4b17-9074-eb3c56037afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:27.718639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cc09b-752d-4b17-9074-eb3c56037afc admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:27.719420 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5cc09b-752d-4b17-9074-eb3c56037afc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:27.721460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5cc09b-752d-4b17-9074-eb3c56037afc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:27.729562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80371e8f-1b2e-4c49-bb10-f41ea76d5b34 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:27.729562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:27.755880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 928/1851] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:27 2022] PUT /identity/v3/projects/0182c426a5a14ef7825c364740ead0ed/users/3abc9cd378424bb1a7d4b38cf21fd0fb/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:27.771897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80371e8f-1b2e-4c49-bb10-f41ea76d5b34 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:27.772846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 924/1852] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:27 2022] POST /identity/v3/users => generated 304 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:27.864855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cc09b-752d-4b17-9074-eb3c56037afc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:27.868379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea5cc09b-752d-4b17-9074-eb3c56037afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:27.901630 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cc09b-752d-4b17-9074-eb3c56037afc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:28.028889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80371e8f-1b2e-4c49-bb10-f41ea76d5b34 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:28.034086 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2b253-f888-4c0d-a69f-32514a8c5bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:28.034829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2b253-f888-4c0d-a69f-32514a8c5bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:28.036222 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2b253-f888-4c0d-a69f-32514a8c5bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:28.036525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2b253-f888-4c0d-a69f-32514a8c5bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:28.037830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2b253-f888-4c0d-a69f-32514a8c5bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:28.057361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2b253-f888-4c0d-a69f-32514a8c5bfd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:28.080276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 929/1853] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:28.080809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a73c8f-0e86-4c00-92c5-91df3b4da879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:28.081834 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a73c8f-0e86-4c00-92c5-91df3b4da879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:28.082845 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a73c8f-0e86-4c00-92c5-91df3b4da879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:28.085625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a73c8f-0e86-4c00-92c5-91df3b4da879 admin admin] PATH_INFO: `/v3/users/c3138a6c6cf94431b916ab30ba3eda81` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:28.090819 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2b253-f888-4c0d-a69f-32514a8c5bfd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:28.096525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a73c8f-0e86-4c00-92c5-91df3b4da879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3138a6c6cf94431b916ab30ba3eda81)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:28.099929 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a73c8f-0e86-4c00-92c5-91df3b4da879 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:28.152325 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:28.155068 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2b253-f888-4c0d-a69f-32514a8c5bfd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:28.183486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcedfb-e509-43de-927a-3f21577ef352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:28.201663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcedfb-e509-43de-927a-3f21577ef352 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:28.203426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcedfb-e509-43de-927a-3f21577ef352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:28.203774 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcedfb-e509-43de-927a-3f21577ef352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:28.207614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcedfb-e509-43de-927a-3f21577ef352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:28.213914 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcedfb-e509-43de-927a-3f21577ef352 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:28.244217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-98a73c8f-0e86-4c00-92c5-91df3b4da879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3138a6c6cf94431b916ab30ba3eda81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:28.246262 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-98a73c8f-0e86-4c00-92c5-91df3b4da879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3138a6c6cf94431b916ab30ba3eda81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:28.265452 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-98a73c8f-0e86-4c00-92c5-91df3b4da879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3138a6c6cf94431b916ab30ba3eda81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:28.270347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 930/1854] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:28.274603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcedfb-e509-43de-927a-3f21577ef352 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:28.296009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-98a73c8f-0e86-4c00-92c5-91df3b4da879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3138a6c6cf94431b916ab30ba3eda81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:28.320921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcedfb-e509-43de-927a-3f21577ef352 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:28.321083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-98a73c8f-0e86-4c00-92c5-91df3b4da879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3138a6c6cf94431b916ab30ba3eda81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:28.332787 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 925/1855] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:27 2022] DELETE /identity/v3/users/c3138a6c6cf94431b916ab30ba3eda81 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:28.334708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86d26d-1323-479c-9c37-28a8b99d8e1a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:28.335488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86d26d-1323-479c-9c37-28a8b99d8e1a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:28.335812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86d26d-1323-479c-9c37-28a8b99d8e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:28.348990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e448b-2465-44d7-b25c-cc30d325ac3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:28.349884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e448b-2465-44d7-b25c-cc30d325ac3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:28.350171 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e448b-2465-44d7-b25c-cc30d325ac3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:28.351021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e448b-2465-44d7-b25c-cc30d325ac3c admin admin] PATH_INFO: `/v3/projects/64a77d9bcd1b4547b57a6180c146ae76/users/02ee142edff94e929e38ca630eaf3938/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:28.366402 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0f86d26d-1323-479c-9c37-28a8b99d8e1a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:28.369124 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f86d26d-1323-479c-9c37-28a8b99d8e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:28.395289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e448b-2465-44d7-b25c-cc30d325ac3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a77d9bcd1b4547b57a6180c146ae76, user_id=02ee142edff94e929e38ca630eaf3938, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:28.398575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e448b-2465-44d7-b25c-cc30d325ac3c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:28.441328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 931/1856] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:28 2022] PUT /identity/v3/projects/64a77d9bcd1b4547b57a6180c146ae76/users/02ee142edff94e929e38ca630eaf3938/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:28.459118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52198d7-29d1-42ad-861a-609988997b45 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:28.464102 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52198d7-29d1-42ad-861a-609988997b45 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:28.464525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52198d7-29d1-42ad-861a-609988997b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:28.547248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 926/1857] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:28.556698 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d52198d7-29d1-42ad-861a-609988997b45 None None] MFA Rules not processed for user `02ee142edff94e929e38ca630eaf3938`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:28.560475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d52198d7-29d1-42ad-861a-609988997b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:28.600960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86d26d-1323-479c-9c37-28a8b99d8e1a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:28.602833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f86d26d-1323-479c-9c37-28a8b99d8e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:28.768954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 932/1858] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:28 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:28.773265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52198d7-29d1-42ad-861a-609988997b45 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:28.777282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d52198d7-29d1-42ad-861a-609988997b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:28.860678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86d26d-1323-479c-9c37-28a8b99d8e1a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:28.940759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:29.012960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52198d7-29d1-42ad-861a-609988997b45 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:29.029775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:29.034017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a259065-1562-4b70-b5b4-4300731ea489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:29.034829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a259065-1562-4b70-b5b4-4300731ea489 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:29.035149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a259065-1562-4b70-b5b4-4300731ea489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:29.041893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a259065-1562-4b70-b5b4-4300731ea489 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:29.042930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a259065-1562-4b70-b5b4-4300731ea489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:29.048193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35205f12-7c09-4253-85a5-d510bf38811a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:29.048921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35205f12-7c09-4253-85a5-d510bf38811a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:29.049268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35205f12-7c09-4253-85a5-d510bf38811a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:29.049622 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35205f12-7c09-4253-85a5-d510bf38811a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:29.052520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a259065-1562-4b70-b5b4-4300731ea489 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:29.114960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35205f12-7c09-4253-85a5-d510bf38811a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:29.116525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35205f12-7c09-4253-85a5-d510bf38811a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:29.122346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 933/1859] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:28 2022] POST /identity/v3/projects => generated 407 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:29.123864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a259065-1562-4b70-b5b4-4300731ea489 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:29.128681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 927/1860] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:29.271838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a259065-1562-4b70-b5b4-4300731ea489 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:29.278630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436945b1-6baa-4212-b01e-fd3bf39b81b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:29.279780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436945b1-6baa-4212-b01e-fd3bf39b81b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:29.280011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436945b1-6baa-4212-b01e-fd3bf39b81b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:29.280256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436945b1-6baa-4212-b01e-fd3bf39b81b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:29.281243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436945b1-6baa-4212-b01e-fd3bf39b81b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:29.283745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436945b1-6baa-4212-b01e-fd3bf39b81b0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:29.314347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 934/1861] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:29 2022] POST /identity/v3/users => generated 310 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:29.320769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35205f12-7c09-4253-85a5-d510bf38811a service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:29.335908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35205f12-7c09-4253-85a5-d510bf38811a service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:29.341296 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396b06b-0da5-48e9-b242-345c02bfd785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:29.343350 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396b06b-0da5-48e9-b242-345c02bfd785 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:29.343925 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396b06b-0da5-48e9-b242-345c02bfd785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:29.344644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396b06b-0da5-48e9-b242-345c02bfd785 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:29.345482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396b06b-0da5-48e9-b242-345c02bfd785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:29.349187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396b06b-0da5-48e9-b242-345c02bfd785 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:29.374214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 928/1862] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:29.374488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436945b1-6baa-4212-b01e-fd3bf39b81b0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:29.387616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436945b1-6baa-4212-b01e-fd3bf39b81b0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:29.397932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36bf58-156d-4684-a403-5531863d3b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:29.399145 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36bf58-156d-4684-a403-5531863d3b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:29.399463 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36bf58-156d-4684-a403-5531863d3b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:29.399880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36bf58-156d-4684-a403-5531863d3b50 admin admin] PATH_INFO: `/v3/projects/f50c1293f8c042cd8bfdcc642072fa25/users/69d60542777141c19ac2ad3e89c2e71d/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:29.432870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36bf58-156d-4684-a403-5531863d3b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50c1293f8c042cd8bfdcc642072fa25, user_id=69d60542777141c19ac2ad3e89c2e71d, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:29.437487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36bf58-156d-4684-a403-5531863d3b50 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:29.474559 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 935/1863] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:29 2022] PUT /identity/v3/projects/f50c1293f8c042cd8bfdcc642072fa25/users/69d60542777141c19ac2ad3e89c2e71d/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:29.482375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396b06b-0da5-48e9-b242-345c02bfd785 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:29.527537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396b06b-0da5-48e9-b242-345c02bfd785 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:29.672065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08511de1-ef68-46e0-bec7-fb6a7f9c47c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:29.675504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08511de1-ef68-46e0-bec7-fb6a7f9c47c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:29.675963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08511de1-ef68-46e0-bec7-fb6a7f9c47c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:29.676862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08511de1-ef68-46e0-bec7-fb6a7f9c47c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:29.679953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08511de1-ef68-46e0-bec7-fb6a7f9c47c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:29.684398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36bf58-156d-4684-a403-5531863d3b50 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:29.688047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08511de1-ef68-46e0-bec7-fb6a7f9c47c3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:29.691015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-db36bf58-156d-4684-a403-5531863d3b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:29.720126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 929/1864] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:29.723064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08511de1-ef68-46e0-bec7-fb6a7f9c47c3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:29.729156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36bf58-156d-4684-a403-5531863d3b50 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:29.736187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e1756-501a-4ee6-ad0a-5499bb58dd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:29.750442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08511de1-ef68-46e0-bec7-fb6a7f9c47c3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:29.772139 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8c637-bc85-4917-8237-9f5f10aad10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:29.773125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c637-bc85-4917-8237-9f5f10aad10d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:29.773649 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c637-bc85-4917-8237-9f5f10aad10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:29.774452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e1756-501a-4ee6-ad0a-5499bb58dd78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:29.775604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c637-bc85-4917-8237-9f5f10aad10d admin admin] PATH_INFO: `/v3/projects/f50c1293f8c042cd8bfdcc642072fa25/users/69d60542777141c19ac2ad3e89c2e71d/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:29.776235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e1756-501a-4ee6-ad0a-5499bb58dd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:29.776574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e1756-501a-4ee6-ad0a-5499bb58dd78 admin admin] PATH_INFO: `/v3/users/426cc6ea65fb4f90b3fd69bd90cd4287` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:29.779277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e1756-501a-4ee6-ad0a-5499bb58dd78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=426cc6ea65fb4f90b3fd69bd90cd4287)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:29.780443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8c637-bc85-4917-8237-9f5f10aad10d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50c1293f8c042cd8bfdcc642072fa25, user_id=69d60542777141c19ac2ad3e89c2e71d, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:29.783634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e1756-501a-4ee6-ad0a-5499bb58dd78 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:29.788301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8c637-bc85-4917-8237-9f5f10aad10d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:29.820360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 930/1865] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:29 2022] PUT /identity/v3/projects/f50c1293f8c042cd8bfdcc642072fa25/users/69d60542777141c19ac2ad3e89c2e71d/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:29.836513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d3722-a07c-421e-984b-a5fca024a6a0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:29.836725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d3722-a07c-421e-984b-a5fca024a6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:29.840535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d3722-a07c-421e-984b-a5fca024a6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:29.919189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-966d3722-a07c-421e-984b-a5fca024a6a0 None None] MFA Rules not processed for user `69d60542777141c19ac2ad3e89c2e71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:29.920505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-966d3722-a07c-421e-984b-a5fca024a6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:29.937030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e7e1756-501a-4ee6-ad0a-5499bb58dd78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '426cc6ea65fb4f90b3fd69bd90cd4287'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:29.937230 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e7e1756-501a-4ee6-ad0a-5499bb58dd78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '426cc6ea65fb4f90b3fd69bd90cd4287'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:29.937914 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e7e1756-501a-4ee6-ad0a-5499bb58dd78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '426cc6ea65fb4f90b3fd69bd90cd4287'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:29.974254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e7e1756-501a-4ee6-ad0a-5499bb58dd78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '426cc6ea65fb4f90b3fd69bd90cd4287'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:29.998859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5e7e1756-501a-4ee6-ad0a-5499bb58dd78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '426cc6ea65fb4f90b3fd69bd90cd4287'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:30.028301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 936/1866] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:29 2022] DELETE /identity/v3/users/426cc6ea65fb4f90b3fd69bd90cd4287 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:30.033575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d0b78-44b7-447d-897c-dc058f48487e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:30.033968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d0b78-44b7-447d-897c-dc058f48487e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:30.034356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d0b78-44b7-447d-897c-dc058f48487e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:30.074594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-657d0b78-44b7-447d-897c-dc058f48487e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:30.077241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-657d0b78-44b7-447d-897c-dc058f48487e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:30.086609 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:30.110663 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 931/1867] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:29 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:30.118968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d3722-a07c-421e-984b-a5fca024a6a0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:30.121664 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-966d3722-a07c-421e-984b-a5fca024a6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:30.276630 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:30.287415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d3722-a07c-421e-984b-a5fca024a6a0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:30.303339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 937/1868] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:30.436889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d557070-22ed-4a44-b767-9c0d6a1d68b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:30.439536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d557070-22ed-4a44-b767-9c0d6a1d68b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:30.442876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d557070-22ed-4a44-b767-9c0d6a1d68b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:30.443184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d557070-22ed-4a44-b767-9c0d6a1d68b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:30.607569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d557070-22ed-4a44-b767-9c0d6a1d68b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:30.611193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d557070-22ed-4a44-b767-9c0d6a1d68b9 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:30.646980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 932/1869] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:30 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:30.671139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d0b78-44b7-447d-897c-dc058f48487e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:30.674989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-657d0b78-44b7-447d-897c-dc058f48487e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:30.861629 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d0b78-44b7-447d-897c-dc058f48487e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:30.870268 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30110cad-6662-4a05-98d3-bd94e7e57ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:30.871454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30110cad-6662-4a05-98d3-bd94e7e57ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:30.871936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30110cad-6662-4a05-98d3-bd94e7e57ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:30.872290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30110cad-6662-4a05-98d3-bd94e7e57ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:30.895725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30110cad-6662-4a05-98d3-bd94e7e57ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:30.897781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30110cad-6662-4a05-98d3-bd94e7e57ed2 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:30.972894 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 938/1870] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:31.086521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d557070-22ed-4a44-b767-9c0d6a1d68b9 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:31.104922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d557070-22ed-4a44-b767-9c0d6a1d68b9 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:31.121508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9670c3d-096c-4476-afbe-36cb51f23f82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:31.138127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9670c3d-096c-4476-afbe-36cb51f23f82 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:31.141200 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9670c3d-096c-4476-afbe-36cb51f23f82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:31.142154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9670c3d-096c-4476-afbe-36cb51f23f82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:31.153772 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b9670c3d-096c-4476-afbe-36cb51f23f82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:31.303421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9670c3d-096c-4476-afbe-36cb51f23f82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:31.306461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9670c3d-096c-4476-afbe-36cb51f23f82 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:31.318521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 933/1871] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:31 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:31.505831 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f9ec6-8c2a-400c-938b-ef9907648235 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:31.506041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f9ec6-8c2a-400c-938b-ef9907648235 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:31.506319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f9ec6-8c2a-400c-938b-ef9907648235 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:31.508040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 939/1872] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 11:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:31.686267 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cba9eb-259e-44ff-81f5-c8c24b82afba None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:31.687267 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cba9eb-259e-44ff-81f5-c8c24b82afba None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:31.688000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cba9eb-259e-44ff-81f5-c8c24b82afba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:31.739488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f7cba9eb-259e-44ff-81f5-c8c24b82afba None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:31.742596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f7cba9eb-259e-44ff-81f5-c8c24b82afba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:31.855670 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ffa5c-4995-4059-8d49-4f9e3d101535 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:31.856739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ffa5c-4995-4059-8d49-4f9e3d101535 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:31.857137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ffa5c-4995-4059-8d49-4f9e3d101535 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:31.860936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 940/1873] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 11:02:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:31.907601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:31.978510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 934/1874] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 11:02:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:32.024110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3921f4f-5cd3-43e1-a494-695feee05abb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:32.024608 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3921f4f-5cd3-43e1-a494-695feee05abb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:32.024845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3921f4f-5cd3-43e1-a494-695feee05abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:32.106960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f3921f4f-5cd3-43e1-a494-695feee05abb None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:32.129019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f3921f4f-5cd3-43e1-a494-695feee05abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:32.160380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 941/1875] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 11:02:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:32.767377 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cba9eb-259e-44ff-81f5-c8c24b82afba None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:32.769437 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f7cba9eb-259e-44ff-81f5-c8c24b82afba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:32.807584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3921f4f-5cd3-43e1-a494-695feee05abb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:32.809522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f3921f4f-5cd3-43e1-a494-695feee05abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:32.812698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cba9eb-259e-44ff-81f5-c8c24b82afba None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:32.825723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:32.827035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:32.827603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:32.828333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] PATH_INFO: `/v3/projects/8cbefe5f6d304d63be1bbd098405866b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:32.833235 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cbefe5f6d304d63be1bbd098405866b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:32.834512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3921f4f-5cd3-43e1-a494-695feee05abb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:32.835821 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:32.844605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6834f-5037-4844-b15d-b55a0816b775 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:32.845755 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6834f-5037-4844-b15d-b55a0816b775 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:32.846049 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6834f-5037-4844-b15d-b55a0816b775 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:32.846333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6834f-5037-4844-b15d-b55a0816b775 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:32.860843 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6834f-5037-4844-b15d-b55a0816b775 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:32.865012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6834f-5037-4844-b15d-b55a0816b775 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:32.934473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cbefe5f6d304d63be1bbd098405866b', 'request_id': 'req-2705e239-de70-48b3-943b-e2c25d2c33ee'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:32.950500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cbefe5f6d304d63be1bbd098405866b', 'request_id': 'req-2705e239-de70-48b3-943b-e2c25d2c33ee'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:32.963170 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cbefe5f6d304d63be1bbd098405866b', 'request_id': 'req-2705e239-de70-48b3-943b-e2c25d2c33ee'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:32.970705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 942/1876] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:32.992976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] The token cache is being invalidate because project 8cbefe5f6d304d63be1bbd098405866b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:32.993437 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:32.995200 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:33.001254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 935/1877] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:32 2022] DELETE /identity/v3/projects/8cbefe5f6d304d63be1bbd098405866b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:33.006783 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6834f-5037-4844-b15d-b55a0816b775 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:33.023548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6834f-5037-4844-b15d-b55a0816b775 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:33.174540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:33.177559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:33.290780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:33.292372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:33.292831 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:33.292933 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] PATH_INFO: `/v3/users/0f3ab50298874bc5a3e52cbde471e335` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:33.299409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f3ab50298874bc5a3e52cbde471e335)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:33.306459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:33.399843 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:33.427732 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f3ab50298874bc5a3e52cbde471e335'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:33.429082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f3ab50298874bc5a3e52cbde471e335'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:33.431642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f3ab50298874bc5a3e52cbde471e335'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:33.501733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f3ab50298874bc5a3e52cbde471e335'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:33.515114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705e239-de70-48b3-943b-e2c25d2c33ee admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:33.527476 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c6213-0d66-4271-ab91-c3ecfa6ce4dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:33.529032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c6213-0d66-4271-ab91-c3ecfa6ce4dd service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:33.529032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c6213-0d66-4271-ab91-c3ecfa6ce4dd service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:33.531212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c6213-0d66-4271-ab91-c3ecfa6ce4dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:33.534900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2f6c6213-0d66-4271-ab91-c3ecfa6ce4dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:33.536794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f3ab50298874bc5a3e52cbde471e335'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:33.569905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c6213-0d66-4271-ab91-c3ecfa6ce4dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:33.572181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c6213-0d66-4271-ab91-c3ecfa6ce4dd service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:33.588465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 943/1878] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:33 2022] DELETE /identity/v3/users/0f3ab50298874bc5a3e52cbde471e335 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:33.614364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 936/1879] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:33.726136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:33.729003 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:33.931011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a37be3-6be3-4e95-8213-d18b6d7e9a7b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:33.946390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccb915-d319-46c1-b0c0-01cb678c3871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:33.948940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccb915-d319-46c1-b0c0-01cb678c3871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:33.950278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccb915-d319-46c1-b0c0-01cb678c3871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:33.954869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccb915-d319-46c1-b0c0-01cb678c3871 admin admin] PATH_INFO: `/v3/projects/5f9695f5a86744e39f11b2fc4b9b20df` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:33.959876 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccb915-d319-46c1-b0c0-01cb678c3871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f9695f5a86744e39f11b2fc4b9b20df)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:33.968842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccb915-d319-46c1-b0c0-01cb678c3871 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:34.106525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0fccb915-d319-46c1-b0c0-01cb678c3871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f9695f5a86744e39f11b2fc4b9b20df', 'request_id': 'req-0fccb915-d319-46c1-b0c0-01cb678c3871'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:34.120167 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0fccb915-d319-46c1-b0c0-01cb678c3871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f9695f5a86744e39f11b2fc4b9b20df', 'request_id': 'req-0fccb915-d319-46c1-b0c0-01cb678c3871'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:34.177523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0fccb915-d319-46c1-b0c0-01cb678c3871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f9695f5a86744e39f11b2fc4b9b20df', 'request_id': 'req-0fccb915-d319-46c1-b0c0-01cb678c3871'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:34.218742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0fccb915-d319-46c1-b0c0-01cb678c3871 admin admin] The token cache is being invalidate because project 5f9695f5a86744e39f11b2fc4b9b20df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:34.219142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0fccb915-d319-46c1-b0c0-01cb678c3871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:34.219887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0fccb915-d319-46c1-b0c0-01cb678c3871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:34.227126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 944/1880] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:33 2022] DELETE /identity/v3/projects/5f9695f5a86744e39f11b2fc4b9b20df => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:34.227993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c6213-0d66-4271-ab91-c3ecfa6ce4dd service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:34.243479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c6213-0d66-4271-ab91-c3ecfa6ce4dd service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:34.378941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:34.398509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:34.400187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:34.400862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:34.401656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] PATH_INFO: `/v3/users/739a7c8b5b414f0eaa061bb4e8c3871f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:34.403360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=739a7c8b5b414f0eaa061bb4e8c3871f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:34.421423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:34.537092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '739a7c8b5b414f0eaa061bb4e8c3871f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:34.537800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '739a7c8b5b414f0eaa061bb4e8c3871f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:34.562742 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '739a7c8b5b414f0eaa061bb4e8c3871f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:34.596438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '739a7c8b5b414f0eaa061bb4e8c3871f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:34.604968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '739a7c8b5b414f0eaa061bb4e8c3871f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:34.615210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 937/1881] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:34 2022] DELETE /identity/v3/users/739a7c8b5b414f0eaa061bb4e8c3871f => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:34.616533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6ac64-cded-4bb8-bde3-3f88572b93ed None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:34.617055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6ac64-cded-4bb8-bde3-3f88572b93ed None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:34.617315 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6ac64-cded-4bb8-bde3-3f88572b93ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:34.704036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8bb6ac64-cded-4bb8-bde3-3f88572b93ed None None] MFA Rules not processed for user `3abc9cd378424bb1a7d4b38cf21fd0fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:34.706777 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8bb6ac64-cded-4bb8-bde3-3f88572b93ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:34.757892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:34.876108 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 945/1882] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:34.883639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:34.886493 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:34.926170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6ac64-cded-4bb8-bde3-3f88572b93ed None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:34.928161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8bb6ac64-cded-4bb8-bde3-3f88572b93ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:34.966335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8af45c-b695-4685-8e75-2435e15c8402 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:34.970864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e086d-015e-4031-98ca-a30de4fff3f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:34.971736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e086d-015e-4031-98ca-a30de4fff3f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:34.972025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e086d-015e-4031-98ca-a30de4fff3f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:34.972452 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e086d-015e-4031-98ca-a30de4fff3f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:34.975369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ba7e086d-015e-4031-98ca-a30de4fff3f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:35.017047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6ac64-cded-4bb8-bde3-3f88572b93ed None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.023371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.024283 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.024566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.024853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] PATH_INFO: `/v3/projects/5890071aeae048548df3f156158d4981` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.026044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5890071aeae048548df3f156158d4981)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.028398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.047546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e086d-015e-4031-98ca-a30de4fff3f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.049209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e086d-015e-4031-98ca-a30de4fff3f3 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.060067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 938/1883] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:35.086355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5890071aeae048548df3f156158d4981', 'request_id': 'req-5bb4616b-7765-4743-b09e-44e41a182e36'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:35.094684 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5890071aeae048548df3f156158d4981', 'request_id': 'req-5bb4616b-7765-4743-b09e-44e41a182e36'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:35.103249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5890071aeae048548df3f156158d4981', 'request_id': 'req-5bb4616b-7765-4743-b09e-44e41a182e36'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:35.118588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] The token cache is being invalidate because project 5890071aeae048548df3f156158d4981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:35.119291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:35.120254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:35.122579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 946/1884] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:34 2022] DELETE /identity/v3/projects/5890071aeae048548df3f156158d4981 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:35.127551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a929e5-dcde-40ea-b69c-1dae260a4fdb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.127857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a929e5-dcde-40ea-b69c-1dae260a4fdb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.128197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a929e5-dcde-40ea-b69c-1dae260a4fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.151864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-88a929e5-dcde-40ea-b69c-1dae260a4fdb None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:35.153349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-88a929e5-dcde-40ea-b69c-1dae260a4fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:35.158828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.160674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:35.184672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:35.264827 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 939/1885] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:35.265481 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:35.266775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc69285-9f4a-4f8e-95be-c379e0a3ec3c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.267068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc69285-9f4a-4f8e-95be-c379e0a3ec3c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.268254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4616b-7765-4743-b09e-44e41a182e36 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.268700 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc69285-9f4a-4f8e-95be-c379e0a3ec3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.278588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.279766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.280776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.281826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db admin admin] PATH_INFO: `/v3/projects/2095f76e57644518a4de181ee522416d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.284667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db admin admin] RBAC: Authorizing `identity:delete_project(project_id=2095f76e57644518a4de181ee522416d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.287056 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.317096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1bc69285-9f4a-4f8e-95be-c379e0a3ec3c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:35.322247 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1bc69285-9f4a-4f8e-95be-c379e0a3ec3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:35.343464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 940/1886] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:35.348198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc69285-9f4a-4f8e-95be-c379e0a3ec3c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.368288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc69285-9f4a-4f8e-95be-c379e0a3ec3c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.385878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2095f76e57644518a4de181ee522416d', 'request_id': 'req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:35.395078 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2095f76e57644518a4de181ee522416d', 'request_id': 'req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:35.404036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2095f76e57644518a4de181ee522416d', 'request_id': 'req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:35.431289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db admin admin] The token cache is being invalidate because project 2095f76e57644518a4de181ee522416d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:35.431621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:35.432533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3a912544-8aa8-4ac5-8a4d-cf0fc07db5db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:35.434747 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 947/1887] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:35 2022] DELETE /identity/v3/projects/2095f76e57644518a4de181ee522416d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:35.439356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07605654-8c21-4b59-aead-55f5a05985d3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.439640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07605654-8c21-4b59-aead-55f5a05985d3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.439992 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07605654-8c21-4b59-aead-55f5a05985d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.450677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19f64d-d69d-4d81-a043-f3dc101ec970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.451597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19f64d-d69d-4d81-a043-f3dc101ec970 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.451995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19f64d-d69d-4d81-a043-f3dc101ec970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.452339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19f64d-d69d-4d81-a043-f3dc101ec970 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.453146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19f64d-d69d-4d81-a043-f3dc101ec970 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:35.453521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19f64d-d69d-4d81-a043-f3dc101ec970 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.455466 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19f64d-d69d-4d81-a043-f3dc101ec970 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.464732 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-07605654-8c21-4b59-aead-55f5a05985d3 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:35.466094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-07605654-8c21-4b59-aead-55f5a05985d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:35.470489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 941/1888] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:35.474879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19f64d-d69d-4d81-a043-f3dc101ec970 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.479708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bf19f64d-d69d-4d81-a043-f3dc101ec970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:35.493704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 948/1889] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:35.497012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66b736-8c35-4db8-ba89-00d88236bfe0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.497343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66b736-8c35-4db8-ba89-00d88236bfe0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.497590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66b736-8c35-4db8-ba89-00d88236bfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.515079 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19f64d-d69d-4d81-a043-f3dc101ec970 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.525824 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cacdc2-9c91-4a8b-9fac-d9f1eda17a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.526881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cacdc2-9c91-4a8b-9fac-d9f1eda17a80 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.527199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cacdc2-9c91-4a8b-9fac-d9f1eda17a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.527485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cacdc2-9c91-4a8b-9fac-d9f1eda17a80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.528367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cacdc2-9c91-4a8b-9fac-d9f1eda17a80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.529556 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3e66b736-8c35-4db8-ba89-00d88236bfe0 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:35.530270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cacdc2-9c91-4a8b-9fac-d9f1eda17a80 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.531202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3e66b736-8c35-4db8-ba89-00d88236bfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:35.539605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 949/1890] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:35.544014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66b736-8c35-4db8-ba89-00d88236bfe0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.554592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66b736-8c35-4db8-ba89-00d88236bfe0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.558779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 942/1891] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:35 2022] POST /identity/v3/projects => generated 387 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:35.559224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da898f0-feee-47e3-8578-de03b35bfcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.559899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da898f0-feee-47e3-8578-de03b35bfcc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.560187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da898f0-feee-47e3-8578-de03b35bfcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.560466 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da898f0-feee-47e3-8578-de03b35bfcc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.561216 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da898f0-feee-47e3-8578-de03b35bfcc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:35.561510 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da898f0-feee-47e3-8578-de03b35bfcc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.563105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da898f0-feee-47e3-8578-de03b35bfcc7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.563222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cacdc2-9c91-4a8b-9fac-d9f1eda17a80 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.579148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 950/1892] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:02:35.580087 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da898f0-feee-47e3-8578-de03b35bfcc7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.589605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da898f0-feee-47e3-8578-de03b35bfcc7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.633218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cacdc2-9c91-4a8b-9fac-d9f1eda17a80 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.637964 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed3840-614b-4757-85d9-a5a41e70760d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.638635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed3840-614b-4757-85d9-a5a41e70760d admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.638942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed3840-614b-4757-85d9-a5a41e70760d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.639323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed3840-614b-4757-85d9-a5a41e70760d admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.640078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed3840-614b-4757-85d9-a5a41e70760d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.642023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed3840-614b-4757-85d9-a5a41e70760d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.656606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902a62c-7b2e-4ca2-839e-0f0dbddf0989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.657335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902a62c-7b2e-4ca2-839e-0f0dbddf0989 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.657614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902a62c-7b2e-4ca2-839e-0f0dbddf0989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.657889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902a62c-7b2e-4ca2-839e-0f0dbddf0989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.658580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902a62c-7b2e-4ca2-839e-0f0dbddf0989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.660513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902a62c-7b2e-4ca2-839e-0f0dbddf0989 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.663598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 943/1893] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:35 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:35.667340 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed3840-614b-4757-85d9-a5a41e70760d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.677052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed3840-614b-4757-85d9-a5a41e70760d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.682557 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788e986-d69b-4467-a275-76a268c39ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.683281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788e986-d69b-4467-a275-76a268c39ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.683549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788e986-d69b-4467-a275-76a268c39ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.683858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788e986-d69b-4467-a275-76a268c39ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.684601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788e986-d69b-4467-a275-76a268c39ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.686189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788e986-d69b-4467-a275-76a268c39ee5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.693575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 951/1894] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:35 2022] POST /identity/v3/projects => generated 389 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:35.695232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902a62c-7b2e-4ca2-839e-0f0dbddf0989 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.698060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 944/1895] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:35.701835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788e986-d69b-4467-a275-76a268c39ee5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.712255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788e986-d69b-4467-a275-76a268c39ee5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.720668 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:35.750595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:35.771243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902a62c-7b2e-4ca2-839e-0f0dbddf0989 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.777107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dacc67-51b7-4573-bd5f-faf811faeb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.778131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dacc67-51b7-4573-bd5f-faf811faeb3b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.778610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dacc67-51b7-4573-bd5f-faf811faeb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.779018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dacc67-51b7-4573-bd5f-faf811faeb3b admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.779977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dacc67-51b7-4573-bd5f-faf811faeb3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.782428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dacc67-51b7-4573-bd5f-faf811faeb3b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.787173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2516e-54b9-4e0e-9219-b7dc71dc96bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.787921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2516e-54b9-4e0e-9219-b7dc71dc96bd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.788214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2516e-54b9-4e0e-9219-b7dc71dc96bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.788492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2516e-54b9-4e0e-9219-b7dc71dc96bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.789227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2516e-54b9-4e0e-9219-b7dc71dc96bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.790974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2516e-54b9-4e0e-9219-b7dc71dc96bd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.807487 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 945/1896] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:35.808393 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2516e-54b9-4e0e-9219-b7dc71dc96bd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.810043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 952/1897] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:35 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:35.813638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dacc67-51b7-4573-bd5f-faf811faeb3b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.818421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2516e-54b9-4e0e-9219-b7dc71dc96bd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.824412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dacc67-51b7-4573-bd5f-faf811faeb3b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.824846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af098a3-fc8a-42d9-9ce9-948ecb6e1463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.825738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af098a3-fc8a-42d9-9ce9-948ecb6e1463 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.826021 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af098a3-fc8a-42d9-9ce9-948ecb6e1463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.826307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af098a3-fc8a-42d9-9ce9-948ecb6e1463 admin admin] PATH_INFO: `/v3/projects/e412686472cd4198967259c54daf5c18/users/a452d3d177b14bd886a07f8532e67272/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.830283 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb32e2-10c2-49af-90f1-1450a8de0185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.831061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb32e2-10c2-49af-90f1-1450a8de0185 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.831713 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb32e2-10c2-49af-90f1-1450a8de0185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.831904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb32e2-10c2-49af-90f1-1450a8de0185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.832862 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb32e2-10c2-49af-90f1-1450a8de0185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.834649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb32e2-10c2-49af-90f1-1450a8de0185 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.848980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 953/1898] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:35.849813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb32e2-10c2-49af-90f1-1450a8de0185 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.857394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af098a3-fc8a-42d9-9ce9-948ecb6e1463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e412686472cd4198967259c54daf5c18, user_id=a452d3d177b14bd886a07f8532e67272, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.862010 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af098a3-fc8a-42d9-9ce9-948ecb6e1463 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.862450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb32e2-10c2-49af-90f1-1450a8de0185 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.869808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc1cf2-96b7-4dc6-9ca5-1a5c6eb1ac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:35.872170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc1cf2-96b7-4dc6-9ca5-1a5c6eb1ac0c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.872629 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc1cf2-96b7-4dc6-9ca5-1a5c6eb1ac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.873118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc1cf2-96b7-4dc6-9ca5-1a5c6eb1ac0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.874381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc1cf2-96b7-4dc6-9ca5-1a5c6eb1ac0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:35.878066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc1cf2-96b7-4dc6-9ca5-1a5c6eb1ac0c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:35.885923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 946/1899] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:35 2022] PUT /identity/v3/projects/e412686472cd4198967259c54daf5c18/users/a452d3d177b14bd886a07f8532e67272/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:35.896839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58100d-3c4b-4117-8c07-481a6e3d0081 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:35.897375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58100d-3c4b-4117-8c07-481a6e3d0081 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:35.897768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58100d-3c4b-4117-8c07-481a6e3d0081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:35.904943 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 954/1900] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:35.909192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc1cf2-96b7-4dc6-9ca5-1a5c6eb1ac0c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:35.925789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc1cf2-96b7-4dc6-9ca5-1a5c6eb1ac0c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:35.938190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0a58100d-3c4b-4117-8c07-481a6e3d0081 None None] MFA Rules not processed for user `a452d3d177b14bd886a07f8532e67272`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:35.939570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0a58100d-3c4b-4117-8c07-481a6e3d0081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:36.004858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af88fc7-881e-4852-9e6d-820debfd6358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.005796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af88fc7-881e-4852-9e6d-820debfd6358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.006137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af88fc7-881e-4852-9e6d-820debfd6358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.006458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af88fc7-881e-4852-9e6d-820debfd6358 admin admin] PATH_INFO: `/v3/projects/25381fdcc1434328b3d081d5122f0b87/users/b722ead631ea441e994fea4f17bc2924/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.023898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af88fc7-881e-4852-9e6d-820debfd6358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25381fdcc1434328b3d081d5122f0b87, user_id=b722ead631ea441e994fea4f17bc2924, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.024059 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 947/1901] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:36.027285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af88fc7-881e-4852-9e6d-820debfd6358 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.028198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58100d-3c4b-4117-8c07-481a6e3d0081 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.040924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58100d-3c4b-4117-8c07-481a6e3d0081 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.045374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 955/1902] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:35 2022] PUT /identity/v3/projects/25381fdcc1434328b3d081d5122f0b87/users/b722ead631ea441e994fea4f17bc2924/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:36.058801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c21123-8fb4-4791-849a-ce4ed4caa09d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.059089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c21123-8fb4-4791-849a-ce4ed4caa09d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.059368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c21123-8fb4-4791-849a-ce4ed4caa09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.084665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d0c21123-8fb4-4791-849a-ce4ed4caa09d None None] MFA Rules not processed for user `b722ead631ea441e994fea4f17bc2924`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:36.085978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d0c21123-8fb4-4791-849a-ce4ed4caa09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:36.098265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e94dc0-fc5a-4c7a-8611-f089ed08511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.098933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e94dc0-fc5a-4c7a-8611-f089ed08511f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.099217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e94dc0-fc5a-4c7a-8611-f089ed08511f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.099497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e94dc0-fc5a-4c7a-8611-f089ed08511f admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.100226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e94dc0-fc5a-4c7a-8611-f089ed08511f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.102658 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e94dc0-fc5a-4c7a-8611-f089ed08511f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.122678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:36.130710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 948/1903] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:36 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:36.135523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e94dc0-fc5a-4c7a-8611-f089ed08511f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.153590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 956/1904] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:36.157828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c21123-8fb4-4791-849a-ce4ed4caa09d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.168334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c21123-8fb4-4791-849a-ce4ed4caa09d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.194543 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:36.198905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e94dc0-fc5a-4c7a-8611-f089ed08511f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.204202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214578b2-48b5-4e93-bdf0-ce9dbcbb3cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.204969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214578b2-48b5-4e93-bdf0-ce9dbcbb3cfb admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.205260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214578b2-48b5-4e93-bdf0-ce9dbcbb3cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.205541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214578b2-48b5-4e93-bdf0-ce9dbcbb3cfb admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.206237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214578b2-48b5-4e93-bdf0-ce9dbcbb3cfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.208280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214578b2-48b5-4e93-bdf0-ce9dbcbb3cfb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.230846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 949/1905] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:36 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:36.231019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a3025-b0b2-4a87-9d2d-7852d7deeccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.231019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a3025-b0b2-4a87-9d2d-7852d7deeccc admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.231019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a3025-b0b2-4a87-9d2d-7852d7deeccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.231019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a3025-b0b2-4a87-9d2d-7852d7deeccc admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.231355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a3025-b0b2-4a87-9d2d-7852d7deeccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.232154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214578b2-48b5-4e93-bdf0-ce9dbcbb3cfb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.232331 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a3025-b0b2-4a87-9d2d-7852d7deeccc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.243005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214578b2-48b5-4e93-bdf0-ce9dbcbb3cfb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.247994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf7571-1ad5-403f-a535-f299b9092422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.248775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf7571-1ad5-403f-a535-f299b9092422 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.249089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf7571-1ad5-403f-a535-f299b9092422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.249398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf7571-1ad5-403f-a535-f299b9092422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.250215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf7571-1ad5-403f-a535-f299b9092422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.251961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf7571-1ad5-403f-a535-f299b9092422 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.258214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 957/1906] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:36 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:36.262162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a3025-b0b2-4a87-9d2d-7852d7deeccc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.267298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 950/1907] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:36.268358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf7571-1ad5-403f-a535-f299b9092422 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.278164 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf7571-1ad5-403f-a535-f299b9092422 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.329231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a3025-b0b2-4a87-9d2d-7852d7deeccc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.334047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d79db-b28f-481e-b683-e620db624a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.335238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d79db-b28f-481e-b683-e620db624a35 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.335522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d79db-b28f-481e-b683-e620db624a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.335762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67e67d-c7f4-4168-b06f-1283b7197d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.335872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d79db-b28f-481e-b683-e620db624a35 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.336513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d79db-b28f-481e-b683-e620db624a35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.336654 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67e67d-c7f4-4168-b06f-1283b7197d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.336938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67e67d-c7f4-4168-b06f-1283b7197d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.337216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67e67d-c7f4-4168-b06f-1283b7197d29 admin admin] PATH_INFO: `/v3/projects/d31e114afb8a4dd0ab45fe0b6f4777c9/users/9a0ea498ef5d4cc49266e0800e0fd5fa/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.338434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d79db-b28f-481e-b683-e620db624a35 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.352956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67e67d-c7f4-4168-b06f-1283b7197d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d31e114afb8a4dd0ab45fe0b6f4777c9, user_id=9a0ea498ef5d4cc49266e0800e0fd5fa, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.355970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67e67d-c7f4-4168-b06f-1283b7197d29 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.359133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 958/1908] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:36 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:36.363305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d79db-b28f-481e-b683-e620db624a35 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.369245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 951/1909] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:36 2022] PUT /identity/v3/projects/d31e114afb8a4dd0ab45fe0b6f4777c9/users/9a0ea498ef5d4cc49266e0800e0fd5fa/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:36.373483 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67e67d-c7f4-4168-b06f-1283b7197d29 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.373614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d79db-b28f-481e-b683-e620db624a35 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.440584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf93abe-4fb5-4754-a10b-4d2a26e040d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.441270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf93abe-4fb5-4754-a10b-4d2a26e040d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.441552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf93abe-4fb5-4754-a10b-4d2a26e040d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.441804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67e67d-c7f4-4168-b06f-1283b7197d29 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.441918 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf93abe-4fb5-4754-a10b-4d2a26e040d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.442637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf93abe-4fb5-4754-a10b-4d2a26e040d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.444459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf93abe-4fb5-4754-a10b-4d2a26e040d7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.446847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bcfd3-616a-46c4-bb7e-99c85cc6017f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.447199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bcfd3-616a-46c4-bb7e-99c85cc6017f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.447484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bcfd3-616a-46c4-bb7e-99c85cc6017f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.447794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bcfd3-616a-46c4-bb7e-99c85cc6017f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.448558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0bcfd3-616a-46c4-bb7e-99c85cc6017f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.450222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0bcfd3-616a-46c4-bb7e-99c85cc6017f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.451593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:36.457928 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 959/1910] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:36.460835 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf93abe-4fb5-4754-a10b-4d2a26e040d7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.462980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 952/1911] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:36.466764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bcfd3-616a-46c4-bb7e-99c85cc6017f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.474445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf93abe-4fb5-4754-a10b-4d2a26e040d7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.478281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bcfd3-616a-46c4-bb7e-99c85cc6017f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.480090 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08175928-f854-4fd7-a250-8a77c9d1c09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.480991 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08175928-f854-4fd7-a250-8a77c9d1c09d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.481266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08175928-f854-4fd7-a250-8a77c9d1c09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.481551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08175928-f854-4fd7-a250-8a77c9d1c09d admin admin] PATH_INFO: `/v3/projects/dd8878d9eb10407bbea5d4f80c14877a/users/d04903a261fb4317b2f49b6983dca9d0/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.484953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528d905-2e6c-41b8-a1fe-7e1b8dfd1cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.485889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528d905-2e6c-41b8-a1fe-7e1b8dfd1cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.486173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528d905-2e6c-41b8-a1fe-7e1b8dfd1cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.486593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528d905-2e6c-41b8-a1fe-7e1b8dfd1cdc admin admin] PATH_INFO: `/v3/projects/d31e114afb8a4dd0ab45fe0b6f4777c9/users/9a0ea498ef5d4cc49266e0800e0fd5fa/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.488661 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528d905-2e6c-41b8-a1fe-7e1b8dfd1cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d31e114afb8a4dd0ab45fe0b6f4777c9, user_id=9a0ea498ef5d4cc49266e0800e0fd5fa, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.491358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528d905-2e6c-41b8-a1fe-7e1b8dfd1cdc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.498360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08175928-f854-4fd7-a250-8a77c9d1c09d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd8878d9eb10407bbea5d4f80c14877a, user_id=d04903a261fb4317b2f49b6983dca9d0, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.501081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08175928-f854-4fd7-a250-8a77c9d1c09d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.502977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 953/1912] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:36 2022] PUT /identity/v3/projects/d31e114afb8a4dd0ab45fe0b6f4777c9/users/9a0ea498ef5d4cc49266e0800e0fd5fa/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:36.514621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458af81c-a3b3-4dad-b188-3c857194f84a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.514949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458af81c-a3b3-4dad-b188-3c857194f84a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.515247 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458af81c-a3b3-4dad-b188-3c857194f84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.518807 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 960/1913] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:36 2022] PUT /identity/v3/projects/dd8878d9eb10407bbea5d4f80c14877a/users/d04903a261fb4317b2f49b6983dca9d0/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:36.523161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08175928-f854-4fd7-a250-8a77c9d1c09d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.540422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:36.554048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-458af81c-a3b3-4dad-b188-3c857194f84a None None] MFA Rules not processed for user `9a0ea498ef5d4cc49266e0800e0fd5fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:36.556011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-458af81c-a3b3-4dad-b188-3c857194f84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:36.601869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08175928-f854-4fd7-a250-8a77c9d1c09d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.607131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f903b-73e5-4ef9-be7f-97318613b46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.607837 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f903b-73e5-4ef9-be7f-97318613b46a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.608122 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f903b-73e5-4ef9-be7f-97318613b46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.608768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f903b-73e5-4ef9-be7f-97318613b46a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.609135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f903b-73e5-4ef9-be7f-97318613b46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.610746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f903b-73e5-4ef9-be7f-97318613b46a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.622293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 961/1914] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:36.625912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f903b-73e5-4ef9-be7f-97318613b46a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.636815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f903b-73e5-4ef9-be7f-97318613b46a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.641211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e2a68-bb70-4e96-83f6-ead1f46f7934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.642089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e2a68-bb70-4e96-83f6-ead1f46f7934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.642371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e2a68-bb70-4e96-83f6-ead1f46f7934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.642645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e2a68-bb70-4e96-83f6-ead1f46f7934 admin admin] PATH_INFO: `/v3/projects/dd8878d9eb10407bbea5d4f80c14877a/users/d04903a261fb4317b2f49b6983dca9d0/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.644662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e2a68-bb70-4e96-83f6-ead1f46f7934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd8878d9eb10407bbea5d4f80c14877a, user_id=d04903a261fb4317b2f49b6983dca9d0, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.647851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e2a68-bb70-4e96-83f6-ead1f46f7934 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.654633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 954/1915] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:36.660726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 962/1916] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:36 2022] PUT /identity/v3/projects/dd8878d9eb10407bbea5d4f80c14877a/users/d04903a261fb4317b2f49b6983dca9d0/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:36.666505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458af81c-a3b3-4dad-b188-3c857194f84a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.667925 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-458af81c-a3b3-4dad-b188-3c857194f84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:36.670821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ef069-1b4b-4a7c-9eb7-deec45574467 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.671688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ef069-1b4b-4a7c-9eb7-deec45574467 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.671688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ef069-1b4b-4a7c-9eb7-deec45574467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.698333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-408ef069-1b4b-4a7c-9eb7-deec45574467 None None] MFA Rules not processed for user `d04903a261fb4317b2f49b6983dca9d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:36.699557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-408ef069-1b4b-4a7c-9eb7-deec45574467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:36.729313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458af81c-a3b3-4dad-b188-3c857194f84a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.767731 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 963/1917] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:36.777765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae831496-e3f5-4ac1-aeaa-5e2fe76d118e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.778864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ef069-1b4b-4a7c-9eb7-deec45574467 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.779749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae831496-e3f5-4ac1-aeaa-5e2fe76d118e service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.780035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae831496-e3f5-4ac1-aeaa-5e2fe76d118e service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.780225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-408ef069-1b4b-4a7c-9eb7-deec45574467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:36.780320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae831496-e3f5-4ac1-aeaa-5e2fe76d118e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.792217 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ef069-1b4b-4a7c-9eb7-deec45574467 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:36.795902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ab078-5a78-47fb-8aea-2107fbd37c61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:36.796543 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ab078-5a78-47fb-8aea-2107fbd37c61 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.796829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ab078-5a78-47fb-8aea-2107fbd37c61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.797110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ab078-5a78-47fb-8aea-2107fbd37c61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.806362 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ab078-5a78-47fb-8aea-2107fbd37c61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.807822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ab078-5a78-47fb-8aea-2107fbd37c61 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.817444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 964/1918] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:36.817861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:36.845192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae831496-e3f5-4ac1-aeaa-5e2fe76d118e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:36.846630 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae831496-e3f5-4ac1-aeaa-5e2fe76d118e service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:36.857061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 955/1919] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:36.927547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ab078-5a78-47fb-8aea-2107fbd37c61 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:36.956900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae831496-e3f5-4ac1-aeaa-5e2fe76d118e service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:37.001327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ab078-5a78-47fb-8aea-2107fbd37c61 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:37.009116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e346a-7997-4681-b30e-c012724689f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:37.010070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e346a-7997-4681-b30e-c012724689f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:37.010364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e346a-7997-4681-b30e-c012724689f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:37.010648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e346a-7997-4681-b30e-c012724689f3 admin admin] PATH_INFO: `/v3/users/a452d3d177b14bd886a07f8532e67272` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:37.011929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e346a-7997-4681-b30e-c012724689f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a452d3d177b14bd886a07f8532e67272)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:37.014930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e346a-7997-4681-b30e-c012724689f3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:37.043425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:37.077936 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae831496-e3f5-4ac1-aeaa-5e2fe76d118e service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:37.099207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5f3aa-043c-4194-a8f0-74427d7a5d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:37.100212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f3aa-043c-4194-a8f0-74427d7a5d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:37.101208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f3aa-043c-4194-a8f0-74427d7a5d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:37.101497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f3aa-043c-4194-a8f0-74427d7a5d42 admin admin] PATH_INFO: `/v3/users/b722ead631ea441e994fea4f17bc2924` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:37.103106 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5f3aa-043c-4194-a8f0-74427d7a5d42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b722ead631ea441e994fea4f17bc2924)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:37.105173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5f3aa-043c-4194-a8f0-74427d7a5d42 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:37.127712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f17e346a-7997-4681-b30e-c012724689f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a452d3d177b14bd886a07f8532e67272'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:37.128403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f17e346a-7997-4681-b30e-c012724689f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a452d3d177b14bd886a07f8532e67272'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:37.129178 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f17e346a-7997-4681-b30e-c012724689f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a452d3d177b14bd886a07f8532e67272'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:37.157664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f17e346a-7997-4681-b30e-c012724689f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a452d3d177b14bd886a07f8532e67272'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:37.181162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f17e346a-7997-4681-b30e-c012724689f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a452d3d177b14bd886a07f8532e67272'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:37.215766 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5fd5f3aa-043c-4194-a8f0-74427d7a5d42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b722ead631ea441e994fea4f17bc2924'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:37.216482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5fd5f3aa-043c-4194-a8f0-74427d7a5d42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b722ead631ea441e994fea4f17bc2924'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:37.217251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 965/1920] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:36 2022] DELETE /identity/v3/users/a452d3d177b14bd886a07f8532e67272 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:37.228689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64624415-baee-4aec-ade4-ac2271f56a2d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:37.228983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64624415-baee-4aec-ade4-ac2271f56a2d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:37.229258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64624415-baee-4aec-ade4-ac2271f56a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:37.232453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5fd5f3aa-043c-4194-a8f0-74427d7a5d42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b722ead631ea441e994fea4f17bc2924'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:37.264523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-64624415-baee-4aec-ade4-ac2271f56a2d None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:37.265801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64624415-baee-4aec-ade4-ac2271f56a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:37.282933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5fd5f3aa-043c-4194-a8f0-74427d7a5d42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b722ead631ea441e994fea4f17bc2924'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:37.303560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5fd5f3aa-043c-4194-a8f0-74427d7a5d42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b722ead631ea441e994fea4f17bc2924'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:37.317677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 956/1921] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:36 2022] DELETE /identity/v3/users/b722ead631ea441e994fea4f17bc2924 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:37.322020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6afb9d-0cc0-4990-b2bc-0c6cd2258dc5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:37.322310 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6afb9d-0cc0-4990-b2bc-0c6cd2258dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:37.322590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6afb9d-0cc0-4990-b2bc-0c6cd2258dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:37.341903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6a6afb9d-0cc0-4990-b2bc-0c6cd2258dc5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:37.343371 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a6afb9d-0cc0-4990-b2bc-0c6cd2258dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:37.375835 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 966/1922] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:37.386072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64624415-baee-4aec-ade4-ac2271f56a2d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:37.388664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64624415-baee-4aec-ade4-ac2271f56a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:37.403911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:37.436979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 957/1923] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:37.451363 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6afb9d-0cc0-4990-b2bc-0c6cd2258dc5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:37.453187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a6afb9d-0cc0-4990-b2bc-0c6cd2258dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:37.502211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64624415-baee-4aec-ade4-ac2271f56a2d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:37.556121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:37.566396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09363da2-275f-4f05-a389-408b8485e60d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:37.567205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09363da2-275f-4f05-a389-408b8485e60d service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:37.567714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09363da2-275f-4f05-a389-408b8485e60d service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:37.568154 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09363da2-275f-4f05-a389-408b8485e60d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:37.578761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6afb9d-0cc0-4990-b2bc-0c6cd2258dc5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:37.581476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09363da2-275f-4f05-a389-408b8485e60d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:37.583518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09363da2-275f-4f05-a389-408b8485e60d service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:37.585691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d43f2f-81d7-4b2e-9cff-2193aba23c2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:37.586448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d43f2f-81d7-4b2e-9cff-2193aba23c2d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:37.586804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d43f2f-81d7-4b2e-9cff-2193aba23c2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:37.587153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d43f2f-81d7-4b2e-9cff-2193aba23c2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:37.597450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 967/1924] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:37.620322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d43f2f-81d7-4b2e-9cff-2193aba23c2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:37.622359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d43f2f-81d7-4b2e-9cff-2193aba23c2d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:37.637750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 958/1925] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:37.695126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09363da2-275f-4f05-a389-408b8485e60d service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:37.697457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-09363da2-275f-4f05-a389-408b8485e60d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:37.772867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09363da2-275f-4f05-a389-408b8485e60d service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:37.783186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9157e1-3e7c-497b-8bc7-c848e4fc5cec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:37.784124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9157e1-3e7c-497b-8bc7-c848e4fc5cec service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:37.784576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9157e1-3e7c-497b-8bc7-c848e4fc5cec service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:37.785013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9157e1-3e7c-497b-8bc7-c848e4fc5cec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:37.787333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0d9157e1-3e7c-497b-8bc7-c848e4fc5cec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:37.876612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9157e1-3e7c-497b-8bc7-c848e4fc5cec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:37.878168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9157e1-3e7c-497b-8bc7-c848e4fc5cec service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:37.890064 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 968/1926] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:38.212447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d43f2f-81d7-4b2e-9cff-2193aba23c2d service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:38.213908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f4d43f2f-81d7-4b2e-9cff-2193aba23c2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:38.225904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9157e1-3e7c-497b-8bc7-c848e4fc5cec service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:38.227544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0d9157e1-3e7c-497b-8bc7-c848e4fc5cec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:38.245829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9157e1-3e7c-497b-8bc7-c848e4fc5cec service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:38.253036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d43f2f-81d7-4b2e-9cff-2193aba23c2d service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:38.257450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:38.260163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:38.261090 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:38.261894 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] PATH_INFO: `/v3/projects/25381fdcc1434328b3d081d5122f0b87` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:38.264115 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:38.265314 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25381fdcc1434328b3d081d5122f0b87)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:38.266399 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:38.270042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:38.270591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] PATH_INFO: `/v3/projects/e412686472cd4198967259c54daf5c18` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:38.272248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:38.283388 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e412686472cd4198967259c54daf5c18)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:38.291752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:38.400942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25381fdcc1434328b3d081d5122f0b87', 'request_id': 'req-4d94c57a-1f08-45bd-8f5b-96a119504b21'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.408971 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e412686472cd4198967259c54daf5c18', 'request_id': 'req-40118814-35ec-43aa-90c7-c09caee9e1cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.410370 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25381fdcc1434328b3d081d5122f0b87', 'request_id': 'req-4d94c57a-1f08-45bd-8f5b-96a119504b21'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.420055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25381fdcc1434328b3d081d5122f0b87', 'request_id': 'req-4d94c57a-1f08-45bd-8f5b-96a119504b21'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.423138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e412686472cd4198967259c54daf5c18', 'request_id': 'req-40118814-35ec-43aa-90c7-c09caee9e1cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.443953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e412686472cd4198967259c54daf5c18', 'request_id': 'req-40118814-35ec-43aa-90c7-c09caee9e1cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.456422 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] The token cache is being invalidate because project e412686472cd4198967259c54daf5c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:38.459133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] The token cache is being invalidate because project 25381fdcc1434328b3d081d5122f0b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:38.459447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.464184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.466983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.469845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 969/1927] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:38 2022] DELETE /identity/v3/projects/25381fdcc1434328b3d081d5122f0b87 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:38.470813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.473406 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:38.474956 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:38.479747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 959/1928] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:38 2022] DELETE /identity/v3/projects/e412686472cd4198967259c54daf5c18 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:38.481504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:38.495721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:38.514337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:38.636197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94c57a-1f08-45bd-8f5b-96a119504b21 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:38.646382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:38.649433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:38.649734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:38.650116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] PATH_INFO: `/v3/users/d04903a261fb4317b2f49b6983dca9d0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:38.654291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d04903a261fb4317b2f49b6983dca9d0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:38.656510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40118814-35ec-43aa-90c7-c09caee9e1cc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:38.656849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:38.673899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:38.674799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:38.675167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:38.678631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] PATH_INFO: `/v3/users/9a0ea498ef5d4cc49266e0800e0fd5fa` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:38.681554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a0ea498ef5d4cc49266e0800e0fd5fa)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:38.683696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:38.731050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:38.747271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd04903a261fb4317b2f49b6983dca9d0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.748780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd04903a261fb4317b2f49b6983dca9d0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.749361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd04903a261fb4317b2f49b6983dca9d0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.774908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd04903a261fb4317b2f49b6983dca9d0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.784959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd04903a261fb4317b2f49b6983dca9d0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.792696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a0ea498ef5d4cc49266e0800e0fd5fa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.793693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a0ea498ef5d4cc49266e0800e0fd5fa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.804546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 970/1929] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:38 2022] DELETE /identity/v3/users/d04903a261fb4317b2f49b6983dca9d0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:38.806656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a0ea498ef5d4cc49266e0800e0fd5fa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.822265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a0ea498ef5d4cc49266e0800e0fd5fa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.832644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a0ea498ef5d4cc49266e0800e0fd5fa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:38.842061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 960/1930] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:38 2022] DELETE /identity/v3/users/9a0ea498ef5d4cc49266e0800e0fd5fa => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:39.227539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:39.230969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:39.248765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:39.250552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:39.307455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19722b-d0a1-4a91-bf77-59136df69ac3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:39.333464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc551de3-b4b0-4163-9e4c-8a3ea856f0da admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:39.338560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a92dfb-3841-431e-8738-09721304547e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:39.339419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a92dfb-3841-431e-8738-09721304547e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.339720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a92dfb-3841-431e-8738-09721304547e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.340000 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a92dfb-3841-431e-8738-09721304547e admin admin] PATH_INFO: `/v3/projects/dd8878d9eb10407bbea5d4f80c14877a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.341171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a92dfb-3841-431e-8738-09721304547e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd8878d9eb10407bbea5d4f80c14877a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:39.343067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a92dfb-3841-431e-8738-09721304547e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:39.360041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:39.378532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499fcb5-0f6a-4fe6-a479-9df000511274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:39.379500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499fcb5-0f6a-4fe6-a479-9df000511274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.379891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499fcb5-0f6a-4fe6-a479-9df000511274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.380255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499fcb5-0f6a-4fe6-a479-9df000511274 admin admin] PATH_INFO: `/v3/projects/d31e114afb8a4dd0ab45fe0b6f4777c9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.381552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499fcb5-0f6a-4fe6-a479-9df000511274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d31e114afb8a4dd0ab45fe0b6f4777c9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:39.384053 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499fcb5-0f6a-4fe6-a479-9df000511274 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:39.397866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-39a92dfb-3841-431e-8738-09721304547e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd8878d9eb10407bbea5d4f80c14877a', 'request_id': 'req-39a92dfb-3841-431e-8738-09721304547e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:39.407060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-39a92dfb-3841-431e-8738-09721304547e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd8878d9eb10407bbea5d4f80c14877a', 'request_id': 'req-39a92dfb-3841-431e-8738-09721304547e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:39.416765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-39a92dfb-3841-431e-8738-09721304547e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd8878d9eb10407bbea5d4f80c14877a', 'request_id': 'req-39a92dfb-3841-431e-8738-09721304547e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:39.429902 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-39a92dfb-3841-431e-8738-09721304547e admin admin] The token cache is being invalidate because project dd8878d9eb10407bbea5d4f80c14877a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:39.430202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-39a92dfb-3841-431e-8738-09721304547e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:39.430862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-39a92dfb-3841-431e-8738-09721304547e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:39.432765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 961/1931] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:39 2022] DELETE /identity/v3/projects/dd8878d9eb10407bbea5d4f80c14877a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:39.437408 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315cf15-5970-42af-bb19-1b8928fe4642 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.437635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315cf15-5970-42af-bb19-1b8928fe4642 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.437920 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315cf15-5970-42af-bb19-1b8928fe4642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.443870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a499fcb5-0f6a-4fe6-a479-9df000511274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd31e114afb8a4dd0ab45fe0b6f4777c9', 'request_id': 'req-a499fcb5-0f6a-4fe6-a479-9df000511274'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:39.450904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a499fcb5-0f6a-4fe6-a479-9df000511274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd31e114afb8a4dd0ab45fe0b6f4777c9', 'request_id': 'req-a499fcb5-0f6a-4fe6-a479-9df000511274'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:39.457222 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a499fcb5-0f6a-4fe6-a479-9df000511274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd31e114afb8a4dd0ab45fe0b6f4777c9', 'request_id': 'req-a499fcb5-0f6a-4fe6-a479-9df000511274'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:39.460463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9315cf15-5970-42af-bb19-1b8928fe4642 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:39.462161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9315cf15-5970-42af-bb19-1b8928fe4642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:39.474059 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a499fcb5-0f6a-4fe6-a479-9df000511274 admin admin] The token cache is being invalidate because project d31e114afb8a4dd0ab45fe0b6f4777c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:39.474415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a499fcb5-0f6a-4fe6-a479-9df000511274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:39.475045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a499fcb5-0f6a-4fe6-a479-9df000511274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:39.476701 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 971/1932] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:39 2022] DELETE /identity/v3/projects/d31e114afb8a4dd0ab45fe0b6f4777c9 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:39.481376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b943785-0fdb-4a8f-912a-7e25356dd2c1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.481647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b943785-0fdb-4a8f-912a-7e25356dd2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.481923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b943785-0fdb-4a8f-912a-7e25356dd2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.496031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:39.503497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2b943785-0fdb-4a8f-912a-7e25356dd2c1 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:39.504765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2b943785-0fdb-4a8f-912a-7e25356dd2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:39.543642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 962/1933] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:39.551603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae7afe-0beb-492c-b817-8896a16080c7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.551980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae7afe-0beb-492c-b817-8896a16080c7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.552332 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae7afe-0beb-492c-b817-8896a16080c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.576687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-14ae7afe-0beb-492c-b817-8896a16080c7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:39.577907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-14ae7afe-0beb-492c-b817-8896a16080c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:39.590761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 972/1934] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:39.591308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 963/1935] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:39.598142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b943785-0fdb-4a8f-912a-7e25356dd2c1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:39.599814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86081804-9e9a-47bc-b7ed-46af20c93e06 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.600215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86081804-9e9a-47bc-b7ed-46af20c93e06 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.601170 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86081804-9e9a-47bc-b7ed-46af20c93e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.609648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b943785-0fdb-4a8f-912a-7e25356dd2c1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:39.618640 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6c6a4-7d52-4822-bf25-aa5e1d989a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:39.619513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6c6a4-7d52-4822-bf25-aa5e1d989a21 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.619828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6c6a4-7d52-4822-bf25-aa5e1d989a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.620106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6c6a4-7d52-4822-bf25-aa5e1d989a21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.621622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6c6a4-7d52-4822-bf25-aa5e1d989a21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:39.622008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6c6a4-7d52-4822-bf25-aa5e1d989a21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:39.623906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6c6a4-7d52-4822-bf25-aa5e1d989a21 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:39.627011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-86081804-9e9a-47bc-b7ed-46af20c93e06 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:39.628251 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-86081804-9e9a-47bc-b7ed-46af20c93e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:39.637715 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 964/1936] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:39.640365 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 973/1937] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:39.641112 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86081804-9e9a-47bc-b7ed-46af20c93e06 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:39.643755 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6c6a4-7d52-4822-bf25-aa5e1d989a21 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:39.651411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86081804-9e9a-47bc-b7ed-46af20c93e06 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:39.654786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6c6a4-7d52-4822-bf25-aa5e1d989a21 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:39.656238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6904977-56f5-4418-9898-4b040a3a5e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:39.656910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6904977-56f5-4418-9898-4b040a3a5e27 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.657192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6904977-56f5-4418-9898-4b040a3a5e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.657469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6904977-56f5-4418-9898-4b040a3a5e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.658374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6904977-56f5-4418-9898-4b040a3a5e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:39.660195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b95b3-74fa-457f-b1fd-51782871636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:39.660771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6904977-56f5-4418-9898-4b040a3a5e27 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:39.661037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b95b3-74fa-457f-b1fd-51782871636f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.661390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b95b3-74fa-457f-b1fd-51782871636f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.661736 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b95b3-74fa-457f-b1fd-51782871636f admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.662533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b95b3-74fa-457f-b1fd-51782871636f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:39.663500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b95b3-74fa-457f-b1fd-51782871636f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:39.665699 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b95b3-74fa-457f-b1fd-51782871636f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:39.683812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 974/1938] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:02:39.685242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b95b3-74fa-457f-b1fd-51782871636f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:39.698386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b95b3-74fa-457f-b1fd-51782871636f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:39.703043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 965/1939] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:39 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:39.704130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6904977-56f5-4418-9898-4b040a3a5e27 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:39.770163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:39.781001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6904977-56f5-4418-9898-4b040a3a5e27 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:39.786390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4c27f-10bf-485d-9515-6d8bcc6eb611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:39.787074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4c27f-10bf-485d-9515-6d8bcc6eb611 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.787413 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4c27f-10bf-485d-9515-6d8bcc6eb611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.787809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4c27f-10bf-485d-9515-6d8bcc6eb611 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.788454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4c27f-10bf-485d-9515-6d8bcc6eb611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:39.789939 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb515d-b3d6-4393-ba25-0e862928fda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:39.790385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4c27f-10bf-485d-9515-6d8bcc6eb611 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:39.790943 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb515d-b3d6-4393-ba25-0e862928fda9 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.792107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb515d-b3d6-4393-ba25-0e862928fda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.792487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb515d-b3d6-4393-ba25-0e862928fda9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.793779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb515d-b3d6-4393-ba25-0e862928fda9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:39.795831 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb515d-b3d6-4393-ba25-0e862928fda9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:39.817986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 966/1940] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:39 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:39.822280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4c27f-10bf-485d-9515-6d8bcc6eb611 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:39.835977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4c27f-10bf-485d-9515-6d8bcc6eb611 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:39.841226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 975/1941] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:39 2022] POST /identity/v3/projects => generated 379 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:39.842220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb515d-b3d6-4393-ba25-0e862928fda9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:39.905861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504a352-ec23-49f4-bcc6-a29644ab1b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:39.906635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504a352-ec23-49f4-bcc6-a29644ab1b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.907252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504a352-ec23-49f4-bcc6-a29644ab1b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.907252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504a352-ec23-49f4-bcc6-a29644ab1b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.908007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504a352-ec23-49f4-bcc6-a29644ab1b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:39.909595 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504a352-ec23-49f4-bcc6-a29644ab1b1c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:39.931416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb515d-b3d6-4393-ba25-0e862928fda9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:39.938123 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9368c9-92ca-4f50-af36-5dbec7d40dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:39.938927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9368c9-92ca-4f50-af36-5dbec7d40dee admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.939310 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9368c9-92ca-4f50-af36-5dbec7d40dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.939886 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9368c9-92ca-4f50-af36-5dbec7d40dee admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.940482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 967/1942] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:39.940864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9368c9-92ca-4f50-af36-5dbec7d40dee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:39.943063 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9368c9-92ca-4f50-af36-5dbec7d40dee admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:39.945593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504a352-ec23-49f4-bcc6-a29644ab1b1c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:39.960293 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504a352-ec23-49f4-bcc6-a29644ab1b1c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:39.971632 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 976/1943] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:39 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:39.972194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995429a-d50a-43a4-83df-a2b4f55bc2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:39.973023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995429a-d50a-43a4-83df-a2b4f55bc2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:39.973377 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995429a-d50a-43a4-83df-a2b4f55bc2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:39.973726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995429a-d50a-43a4-83df-a2b4f55bc2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:39.974622 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995429a-d50a-43a4-83df-a2b4f55bc2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:39.976796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995429a-d50a-43a4-83df-a2b4f55bc2b6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:39.977437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9368c9-92ca-4f50-af36-5dbec7d40dee admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:39.989638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 968/1944] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:39.993805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995429a-d50a-43a4-83df-a2b4f55bc2b6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:39.999252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9368c9-92ca-4f50-af36-5dbec7d40dee admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:40.004322 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136c70b-4b38-456a-b10e-44f6409625e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:40.005002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136c70b-4b38-456a-b10e-44f6409625e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.005366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136c70b-4b38-456a-b10e-44f6409625e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.005663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136c70b-4b38-456a-b10e-44f6409625e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.006444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136c70b-4b38-456a-b10e-44f6409625e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:40.009075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136c70b-4b38-456a-b10e-44f6409625e2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:40.011124 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995429a-d50a-43a4-83df-a2b4f55bc2b6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:40.017651 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a4399-ff7a-4c97-b596-9c8195ad6023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:40.018915 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a4399-ff7a-4c97-b596-9c8195ad6023 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.019287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a4399-ff7a-4c97-b596-9c8195ad6023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.019635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a4399-ff7a-4c97-b596-9c8195ad6023 admin admin] PATH_INFO: `/v3/projects/34d70fd94675477e9e0a1971d9454a7a/users/bdb2f7218b2f463a949b401e9862b19a/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.027161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:40.027437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 977/1945] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:40.029602 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136c70b-4b38-456a-b10e-44f6409625e2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:40.042965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136c70b-4b38-456a-b10e-44f6409625e2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:40.046936 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a4399-ff7a-4c97-b596-9c8195ad6023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d70fd94675477e9e0a1971d9454a7a, user_id=bdb2f7218b2f463a949b401e9862b19a, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:40.048324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a99655-9c28-40cb-8ea0-d94a10755a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:40.049314 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a99655-9c28-40cb-8ea0-d94a10755a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.049555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a99655-9c28-40cb-8ea0-d94a10755a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.049726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a4399-ff7a-4c97-b596-9c8195ad6023 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:40.049829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a99655-9c28-40cb-8ea0-d94a10755a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.050570 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a99655-9c28-40cb-8ea0-d94a10755a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:40.052193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a99655-9c28-40cb-8ea0-d94a10755a12 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:40.064528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 978/1946] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:40.067157 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 969/1947] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:39 2022] PUT /identity/v3/projects/34d70fd94675477e9e0a1971d9454a7a/users/bdb2f7218b2f463a949b401e9862b19a/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:40.077683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a99655-9c28-40cb-8ea0-d94a10755a12 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:40.079078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1261cad-dd32-4f13-b3b4-5f26b4c5bdd6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.079192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a99655-9c28-40cb-8ea0-d94a10755a12 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:40.079366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1261cad-dd32-4f13-b3b4-5f26b4c5bdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.079644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1261cad-dd32-4f13-b3b4-5f26b4c5bdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.111634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d1261cad-dd32-4f13-b3b4-5f26b4c5bdd6 None None] MFA Rules not processed for user `bdb2f7218b2f463a949b401e9862b19a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:40.113525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d1261cad-dd32-4f13-b3b4-5f26b4c5bdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:40.155681 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f18a0-1c9a-485b-b330-d6e61998e602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:40.156591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f18a0-1c9a-485b-b330-d6e61998e602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.156876 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f18a0-1c9a-485b-b330-d6e61998e602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.157154 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f18a0-1c9a-485b-b330-d6e61998e602 admin admin] PATH_INFO: `/v3/projects/04ae031782034c63885157c3be1b139c/users/b89dc9b24b2f4890994ea90928856f74/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.172012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f18a0-1c9a-485b-b330-d6e61998e602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ae031782034c63885157c3be1b139c, user_id=b89dc9b24b2f4890994ea90928856f74, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:40.174694 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f18a0-1c9a-485b-b330-d6e61998e602 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:40.189641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 979/1948] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:40 2022] PUT /identity/v3/projects/04ae031782034c63885157c3be1b139c/users/b89dc9b24b2f4890994ea90928856f74/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:40.198430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 970/1949] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:40.201332 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71948c55-f1ea-47ab-91ce-db9a21c2a050 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.201781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71948c55-f1ea-47ab-91ce-db9a21c2a050 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.202052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71948c55-f1ea-47ab-91ce-db9a21c2a050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.210439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1261cad-dd32-4f13-b3b4-5f26b4c5bdd6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:40.211961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d1261cad-dd32-4f13-b3b4-5f26b4c5bdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:40.232808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-71948c55-f1ea-47ab-91ce-db9a21c2a050 None None] MFA Rules not processed for user `b89dc9b24b2f4890994ea90928856f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:40.234109 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-71948c55-f1ea-47ab-91ce-db9a21c2a050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:40.250281 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:40.282183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1261cad-dd32-4f13-b3b4-5f26b4c5bdd6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:40.321042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 980/1950] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:40.322421 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71948c55-f1ea-47ab-91ce-db9a21c2a050 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:40.338354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b742b6e-0510-4225-96fd-88836d56089a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:40.339010 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b742b6e-0510-4225-96fd-88836d56089a service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.339294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b742b6e-0510-4225-96fd-88836d56089a service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.339573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b742b6e-0510-4225-96fd-88836d56089a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.384873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:40.385555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71948c55-f1ea-47ab-91ce-db9a21c2a050 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:40.390795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d9703-075f-4231-b88d-334e598f51e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:40.391680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d9703-075f-4231-b88d-334e598f51e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.392062 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d9703-075f-4231-b88d-334e598f51e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.392345 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d9703-075f-4231-b88d-334e598f51e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.393046 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d9703-075f-4231-b88d-334e598f51e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:40.394949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d9703-075f-4231-b88d-334e598f51e8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:40.399336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b742b6e-0510-4225-96fd-88836d56089a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:40.403510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b742b6e-0510-4225-96fd-88836d56089a service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:40.410032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 971/1951] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:40.425351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 981/1952] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:40 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:40.426525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b742b6e-0510-4225-96fd-88836d56089a service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:40.436900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b742b6e-0510-4225-96fd-88836d56089a service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:40.492703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5a658-ce61-4844-b136-547477a00a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:40.493376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5a658-ce61-4844-b136-547477a00a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.493659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5a658-ce61-4844-b136-547477a00a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.493957 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5a658-ce61-4844-b136-547477a00a49 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.494636 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5a658-ce61-4844-b136-547477a00a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:40.496836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5a658-ce61-4844-b136-547477a00a49 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:40.520442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 972/1953] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:40 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:40.521617 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d9703-075f-4231-b88d-334e598f51e8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:40.532923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d9703-075f-4231-b88d-334e598f51e8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:40.537698 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ca6aa-38c4-4f61-8665-1bddd663bdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:40.538366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ca6aa-38c4-4f61-8665-1bddd663bdab admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.538647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ca6aa-38c4-4f61-8665-1bddd663bdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.538923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ca6aa-38c4-4f61-8665-1bddd663bdab admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.539679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ca6aa-38c4-4f61-8665-1bddd663bdab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:40.541351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ca6aa-38c4-4f61-8665-1bddd663bdab admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:40.555523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 982/1954] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:40.557175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5a658-ce61-4844-b136-547477a00a49 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:40.571931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5a658-ce61-4844-b136-547477a00a49 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:40.576882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e84a15-280d-4a34-a274-a59f1988ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:40.577844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e84a15-280d-4a34-a274-a59f1988ccf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.578128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e84a15-280d-4a34-a274-a59f1988ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.578398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e84a15-280d-4a34-a274-a59f1988ccf8 admin admin] PATH_INFO: `/v3/projects/fc984b18f5dc4c67bb84f0f9f1769564/users/65ab2d599264454eac3070d3b7f6e506/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.594696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e84a15-280d-4a34-a274-a59f1988ccf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc984b18f5dc4c67bb84f0f9f1769564, user_id=65ab2d599264454eac3070d3b7f6e506, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:40.597416 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e84a15-280d-4a34-a274-a59f1988ccf8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:40.613497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 973/1955] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:40 2022] PUT /identity/v3/projects/fc984b18f5dc4c67bb84f0f9f1769564/users/65ab2d599264454eac3070d3b7f6e506/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:40.617728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ca6aa-38c4-4f61-8665-1bddd663bdab admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:40.629677 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ca6aa-38c4-4f61-8665-1bddd663bdab admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:40.753490 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf41cd9-8a80-4349-a515-743734440a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:40.754175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf41cd9-8a80-4349-a515-743734440a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.754458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf41cd9-8a80-4349-a515-743734440a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.754734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf41cd9-8a80-4349-a515-743734440a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.755482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf41cd9-8a80-4349-a515-743734440a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:40.757676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf41cd9-8a80-4349-a515-743734440a55 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:40.784811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 983/1956] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:40.789302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e84a15-280d-4a34-a274-a59f1988ccf8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:40.805309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e84a15-280d-4a34-a274-a59f1988ccf8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:40.817901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2791550-70fa-4808-bac8-113f948eac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:40.821475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2791550-70fa-4808-bac8-113f948eac2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.821760 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2791550-70fa-4808-bac8-113f948eac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.823482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2791550-70fa-4808-bac8-113f948eac2e admin admin] PATH_INFO: `/v3/projects/fc984b18f5dc4c67bb84f0f9f1769564/users/65ab2d599264454eac3070d3b7f6e506/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.832568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2791550-70fa-4808-bac8-113f948eac2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc984b18f5dc4c67bb84f0f9f1769564, user_id=65ab2d599264454eac3070d3b7f6e506, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:40.839568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2791550-70fa-4808-bac8-113f948eac2e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:40.855462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 974/1957] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:40 2022] PUT /identity/v3/projects/fc984b18f5dc4c67bb84f0f9f1769564/users/65ab2d599264454eac3070d3b7f6e506/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:40.864497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d591aed-8381-4e75-b20f-cc21a4b14fca None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.864786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d591aed-8381-4e75-b20f-cc21a4b14fca None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.865066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d591aed-8381-4e75-b20f-cc21a4b14fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.897853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5d591aed-8381-4e75-b20f-cc21a4b14fca None None] MFA Rules not processed for user `65ab2d599264454eac3070d3b7f6e506`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:40.899071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5d591aed-8381-4e75-b20f-cc21a4b14fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:40.914660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:41.051605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 984/1958] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:41.059639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2791550-70fa-4808-bac8-113f948eac2e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:41.061428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e2791550-70fa-4808-bac8-113f948eac2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:41.217914 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2791550-70fa-4808-bac8-113f948eac2e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:41.229724 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd64d1-5e35-4fa6-a849-2455f5c706af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:41.232928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd64d1-5e35-4fa6-a849-2455f5c706af service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:41.233502 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd64d1-5e35-4fa6-a849-2455f5c706af service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:41.233624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd64d1-5e35-4fa6-a849-2455f5c706af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:41.244517 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd64d1-5e35-4fa6-a849-2455f5c706af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:41.246114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd64d1-5e35-4fa6-a849-2455f5c706af service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:41.254941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d591aed-8381-4e75-b20f-cc21a4b14fca None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:41.258114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 975/1959] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:41.261546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5d591aed-8381-4e75-b20f-cc21a4b14fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:41.385710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d591aed-8381-4e75-b20f-cc21a4b14fca None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:41.401957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:41.403798 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:41.404650 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:41.405359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd64d1-5e35-4fa6-a849-2455f5c706af service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:41.406486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] PATH_INFO: `/v3/users/02ee142edff94e929e38ca630eaf3938` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:41.409820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02ee142edff94e929e38ca630eaf3938)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:41.412666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:41.419320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd64d1-5e35-4fa6-a849-2455f5c706af service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:41.442757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9eb87-f6fb-4bd1-be8c-d4a2e08cbb92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:41.445382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9eb87-f6fb-4bd1-be8c-d4a2e08cbb92 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:41.445672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9eb87-f6fb-4bd1-be8c-d4a2e08cbb92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:41.447292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9eb87-f6fb-4bd1-be8c-d4a2e08cbb92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:41.508116 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:41.513923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02ee142edff94e929e38ca630eaf3938'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:41.514944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02ee142edff94e929e38ca630eaf3938'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:41.516016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02ee142edff94e929e38ca630eaf3938'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:41.536700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02ee142edff94e929e38ca630eaf3938'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:41.543278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9eb87-f6fb-4bd1-be8c-d4a2e08cbb92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:41.545525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9eb87-f6fb-4bd1-be8c-d4a2e08cbb92 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:41.545858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02ee142edff94e929e38ca630eaf3938'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:41.567138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 976/1960] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:41.570440 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 985/1961] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:41 2022] DELETE /identity/v3/users/02ee142edff94e929e38ca630eaf3938 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:41.575526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a005b-5796-40c4-a602-b3176f02d9f7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:41.575840 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a005b-5796-40c4-a602-b3176f02d9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:41.576115 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a005b-5796-40c4-a602-b3176f02d9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:41.595240 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-888a005b-5796-40c4-a602-b3176f02d9f7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:41.596473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-888a005b-5796-40c4-a602-b3176f02d9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:41.687852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 977/1962] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:41.698489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:41.700391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:41.739711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:41.774044 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46916559-7dbf-4b61-87a2-9f7129bbebc6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:41.779152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84729cbc-2a9e-490a-b56f-54683660f6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:41.780240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84729cbc-2a9e-490a-b56f-54683660f6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:41.780638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84729cbc-2a9e-490a-b56f-54683660f6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:41.780950 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84729cbc-2a9e-490a-b56f-54683660f6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:41.793580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84729cbc-2a9e-490a-b56f-54683660f6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:41.796756 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84729cbc-2a9e-490a-b56f-54683660f6b3 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:41.821284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 986/1963] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:43.066811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a005b-5796-40c4-a602-b3176f02d9f7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:43.068281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-888a005b-5796-40c4-a602-b3176f02d9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:43.080656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a005b-5796-40c4-a602-b3176f02d9f7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:43.097681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:43.098984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:43.099273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:43.099734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] PATH_INFO: `/v3/projects/64a77d9bcd1b4547b57a6180c146ae76` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:43.101755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a77d9bcd1b4547b57a6180c146ae76)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:43.103788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:43.172943 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a77d9bcd1b4547b57a6180c146ae76', 'request_id': 'req-3180bea1-a648-401a-bbca-73e82dbb01c3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.181982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a77d9bcd1b4547b57a6180c146ae76', 'request_id': 'req-3180bea1-a648-401a-bbca-73e82dbb01c3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.189817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a77d9bcd1b4547b57a6180c146ae76', 'request_id': 'req-3180bea1-a648-401a-bbca-73e82dbb01c3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.203197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] The token cache is being invalidate because project 64a77d9bcd1b4547b57a6180c146ae76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:43.203493 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.204189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.206176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 978/1964] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:43 2022] DELETE /identity/v3/projects/64a77d9bcd1b4547b57a6180c146ae76 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:43.210582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84729cbc-2a9e-490a-b56f-54683660f6b3 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:43.222501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84729cbc-2a9e-490a-b56f-54683660f6b3 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:43.298649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a15d5-f851-4b36-8552-49c3368b0116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:43.299685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a15d5-f851-4b36-8552-49c3368b0116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:43.299866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a15d5-f851-4b36-8552-49c3368b0116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:43.300150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a15d5-f851-4b36-8552-49c3368b0116 admin admin] PATH_INFO: `/v3/users/69d60542777141c19ac2ad3e89c2e71d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:43.301431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a15d5-f851-4b36-8552-49c3368b0116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69d60542777141c19ac2ad3e89c2e71d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:43.303479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a15d5-f851-4b36-8552-49c3368b0116 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:43.352338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-145a15d5-f851-4b36-8552-49c3368b0116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69d60542777141c19ac2ad3e89c2e71d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.352989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-145a15d5-f851-4b36-8552-49c3368b0116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69d60542777141c19ac2ad3e89c2e71d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.353591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-145a15d5-f851-4b36-8552-49c3368b0116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69d60542777141c19ac2ad3e89c2e71d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.365645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-145a15d5-f851-4b36-8552-49c3368b0116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69d60542777141c19ac2ad3e89c2e71d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.372643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-145a15d5-f851-4b36-8552-49c3368b0116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69d60542777141c19ac2ad3e89c2e71d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.384903 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 987/1965] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:43 2022] DELETE /identity/v3/users/69d60542777141c19ac2ad3e89c2e71d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:43.674607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:43.676198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:43.690671 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180bea1-a648-401a-bbca-73e82dbb01c3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:43.748960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d5359-ce2a-44f6-982e-fcded58620a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:43.749841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d5359-ce2a-44f6-982e-fcded58620a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:43.750122 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d5359-ce2a-44f6-982e-fcded58620a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:43.750397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d5359-ce2a-44f6-982e-fcded58620a3 admin admin] PATH_INFO: `/v3/projects/f50c1293f8c042cd8bfdcc642072fa25` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:43.751582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d5359-ce2a-44f6-982e-fcded58620a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50c1293f8c042cd8bfdcc642072fa25)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:43.753522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d5359-ce2a-44f6-982e-fcded58620a3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:43.755743 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:43.802079 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-035d5359-ce2a-44f6-982e-fcded58620a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50c1293f8c042cd8bfdcc642072fa25', 'request_id': 'req-035d5359-ce2a-44f6-982e-fcded58620a3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.809802 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-035d5359-ce2a-44f6-982e-fcded58620a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50c1293f8c042cd8bfdcc642072fa25', 'request_id': 'req-035d5359-ce2a-44f6-982e-fcded58620a3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.815687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-035d5359-ce2a-44f6-982e-fcded58620a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50c1293f8c042cd8bfdcc642072fa25', 'request_id': 'req-035d5359-ce2a-44f6-982e-fcded58620a3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.827408 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-035d5359-ce2a-44f6-982e-fcded58620a3 admin admin] The token cache is being invalidate because project f50c1293f8c042cd8bfdcc642072fa25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:43.827722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-035d5359-ce2a-44f6-982e-fcded58620a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.828372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-035d5359-ce2a-44f6-982e-fcded58620a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:43.829904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 979/1966] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:43 2022] DELETE /identity/v3/projects/f50c1293f8c042cd8bfdcc642072fa25 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:43.834748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9baf1-60f1-42b2-a118-ba66e96923da None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:43.835040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9baf1-60f1-42b2-a118-ba66e96923da None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:43.835319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9baf1-60f1-42b2-a118-ba66e96923da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:43.853147 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8bf9baf1-60f1-42b2-a118-ba66e96923da None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:43.854355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8bf9baf1-60f1-42b2-a118-ba66e96923da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:43.904022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:43.924430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 988/1967] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:43.933093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece03ec-1ad5-4e4e-8332-fcb6b8dad876 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:43.933385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece03ec-1ad5-4e4e-8332-fcb6b8dad876 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:43.933693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece03ec-1ad5-4e4e-8332-fcb6b8dad876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:43.955288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8ece03ec-1ad5-4e4e-8332-fcb6b8dad876 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:43.957241 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8ece03ec-1ad5-4e4e-8332-fcb6b8dad876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:43.970788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9baf1-60f1-42b2-a118-ba66e96923da None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:43.971733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 980/1968] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:43.980910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9baf1-60f1-42b2-a118-ba66e96923da None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:43.986554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b6f24-7f7f-445f-b485-78b40ce7c508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:43.987329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b6f24-7f7f-445f-b485-78b40ce7c508 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:43.987713 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b6f24-7f7f-445f-b485-78b40ce7c508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:43.988058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b6f24-7f7f-445f-b485-78b40ce7c508 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:43.988864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b6f24-7f7f-445f-b485-78b40ce7c508 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:43.989537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b6f24-7f7f-445f-b485-78b40ce7c508 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:43.991810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b6f24-7f7f-445f-b485-78b40ce7c508 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:44.004667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 989/1969] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:44.009713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece03ec-1ad5-4e4e-8332-fcb6b8dad876 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:44.018949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece03ec-1ad5-4e4e-8332-fcb6b8dad876 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:44.023578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4529-3043-4749-904f-c557936532c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:44.024259 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4529-3043-4749-904f-c557936532c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.024537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4529-3043-4749-904f-c557936532c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.024812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4529-3043-4749-904f-c557936532c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.025500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed4529-3043-4749-904f-c557936532c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:44.027380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed4529-3043-4749-904f-c557936532c7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:44.053929 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 981/1970] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:44 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:44.059300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b6f24-7f7f-445f-b485-78b40ce7c508 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:44.070291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b6f24-7f7f-445f-b485-78b40ce7c508 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:44.139967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f6111-ac56-4a02-a03f-da81197ee7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:44.140650 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f6111-ac56-4a02-a03f-da81197ee7af admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.140937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f6111-ac56-4a02-a03f-da81197ee7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.141216 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f6111-ac56-4a02-a03f-da81197ee7af admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.141917 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f6111-ac56-4a02-a03f-da81197ee7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:44.143931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f6111-ac56-4a02-a03f-da81197ee7af admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:44.169168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 990/1971] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:44 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:44.170594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4529-3043-4749-904f-c557936532c7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:44.182368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4529-3043-4749-904f-c557936532c7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:44.187010 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9274bd3-ddab-4ee9-971e-33fc3e5e0426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:44.187703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9274bd3-ddab-4ee9-971e-33fc3e5e0426 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.187993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9274bd3-ddab-4ee9-971e-33fc3e5e0426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.188257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9274bd3-ddab-4ee9-971e-33fc3e5e0426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.188982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9274bd3-ddab-4ee9-971e-33fc3e5e0426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:44.190685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9274bd3-ddab-4ee9-971e-33fc3e5e0426 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:44.201834 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 982/1972] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:44.205587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f6111-ac56-4a02-a03f-da81197ee7af admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:44.215887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f6111-ac56-4a02-a03f-da81197ee7af admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:44.220522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a23e07-a569-4293-99f2-102c48a105ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:44.221181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a23e07-a569-4293-99f2-102c48a105ca admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.221483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a23e07-a569-4293-99f2-102c48a105ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.221757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a23e07-a569-4293-99f2-102c48a105ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.222469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a23e07-a569-4293-99f2-102c48a105ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:44.224094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a23e07-a569-4293-99f2-102c48a105ca admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:44.234474 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 991/1973] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:44.238520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9274bd3-ddab-4ee9-971e-33fc3e5e0426 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:44.249445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9274bd3-ddab-4ee9-971e-33fc3e5e0426 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:44.254068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5725e-ba1d-4a26-bc27-447a396dc06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:44.254942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5725e-ba1d-4a26-bc27-447a396dc06a admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.255223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5725e-ba1d-4a26-bc27-447a396dc06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.255498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5725e-ba1d-4a26-bc27-447a396dc06a admin admin] PATH_INFO: `/v3/projects/264d23c899ae4b3289da270d559c1759/users/13f645dac60041fe9fd6ea9b20d88ff3/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.269666 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5725e-ba1d-4a26-bc27-447a396dc06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=264d23c899ae4b3289da270d559c1759, user_id=13f645dac60041fe9fd6ea9b20d88ff3, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:44.272374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5725e-ba1d-4a26-bc27-447a396dc06a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:44.284727 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 983/1974] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:44 2022] PUT /identity/v3/projects/264d23c899ae4b3289da270d559c1759/users/13f645dac60041fe9fd6ea9b20d88ff3/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:44.295459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b485b2-6a96-44df-86cc-236bd951474d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.295788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b485b2-6a96-44df-86cc-236bd951474d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.296069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b485b2-6a96-44df-86cc-236bd951474d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.323084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d2b485b2-6a96-44df-86cc-236bd951474d None None] MFA Rules not processed for user `13f645dac60041fe9fd6ea9b20d88ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:44.324261 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d2b485b2-6a96-44df-86cc-236bd951474d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:44.391850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 992/1975] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:44.396077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5725e-ba1d-4a26-bc27-447a396dc06a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:44.456887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5725e-ba1d-4a26-bc27-447a396dc06a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:44.461496 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64befff8-6b4f-4656-aad2-b052b7a721f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:44.462153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64befff8-6b4f-4656-aad2-b052b7a721f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.462440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64befff8-6b4f-4656-aad2-b052b7a721f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.462716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64befff8-6b4f-4656-aad2-b052b7a721f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.463418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64befff8-6b4f-4656-aad2-b052b7a721f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:44.465362 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64befff8-6b4f-4656-aad2-b052b7a721f5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:44.489775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 984/1976] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:44 2022] POST /identity/v3/projects => generated 379 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:44.494480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b485b2-6a96-44df-86cc-236bd951474d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:44.505317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b485b2-6a96-44df-86cc-236bd951474d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:44.550198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:44.563498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a1521-ab6e-4f43-92db-8d63ef66a3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:44.564202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a1521-ab6e-4f43-92db-8d63ef66a3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.564489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a1521-ab6e-4f43-92db-8d63ef66a3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.564767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a1521-ab6e-4f43-92db-8d63ef66a3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.565464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a1521-ab6e-4f43-92db-8d63ef66a3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:44.567416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a1521-ab6e-4f43-92db-8d63ef66a3a8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:44.586805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 993/1977] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:44 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:44.591031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64befff8-6b4f-4656-aad2-b052b7a721f5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:44.604025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64befff8-6b4f-4656-aad2-b052b7a721f5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:44.609182 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fff974-4ee9-4f94-8682-938eda163c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:44.609882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fff974-4ee9-4f94-8682-938eda163c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.610158 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fff974-4ee9-4f94-8682-938eda163c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.610436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fff974-4ee9-4f94-8682-938eda163c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.611155 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fff974-4ee9-4f94-8682-938eda163c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:44.612764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fff974-4ee9-4f94-8682-938eda163c27 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:44.624580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 985/1978] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:44.628303 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a1521-ab6e-4f43-92db-8d63ef66a3a8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:44.638672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a1521-ab6e-4f43-92db-8d63ef66a3a8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:44.643249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16165965-e33c-499a-a316-b25cce87df5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:44.643925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16165965-e33c-499a-a316-b25cce87df5b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.644205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16165965-e33c-499a-a316-b25cce87df5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.644481 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16165965-e33c-499a-a316-b25cce87df5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.645234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16165965-e33c-499a-a316-b25cce87df5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:44.646803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16165965-e33c-499a-a316-b25cce87df5b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:44.657062 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 994/1979] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:44.661028 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fff974-4ee9-4f94-8682-938eda163c27 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:44.666780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:44.672977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fff974-4ee9-4f94-8682-938eda163c27 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:44.678445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22ad3b-90a7-40ed-9cbc-a4256bcd9512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:44.679321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22ad3b-90a7-40ed-9cbc-a4256bcd9512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.679595 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22ad3b-90a7-40ed-9cbc-a4256bcd9512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.679934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22ad3b-90a7-40ed-9cbc-a4256bcd9512 admin admin] PATH_INFO: `/v3/projects/f61e05a59f2b41f6bca7aed76ac258a0/users/15fa41a31d6a434eae4210de5161a9d3/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.694716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22ad3b-90a7-40ed-9cbc-a4256bcd9512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f61e05a59f2b41f6bca7aed76ac258a0, user_id=15fa41a31d6a434eae4210de5161a9d3, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:44.697391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22ad3b-90a7-40ed-9cbc-a4256bcd9512 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:44.710786 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 986/1980] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:44 2022] PUT /identity/v3/projects/f61e05a59f2b41f6bca7aed76ac258a0/users/15fa41a31d6a434eae4210de5161a9d3/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:44.721706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04f8b1-f830-403f-b690-d4f51f586278 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.722078 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04f8b1-f830-403f-b690-d4f51f586278 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.722381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04f8b1-f830-403f-b690-d4f51f586278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.749742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ef04f8b1-f830-403f-b690-d4f51f586278 None None] MFA Rules not processed for user `15fa41a31d6a434eae4210de5161a9d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:44.750948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ef04f8b1-f830-403f-b690-d4f51f586278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:44.819402 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 995/1981] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:44.823777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22ad3b-90a7-40ed-9cbc-a4256bcd9512 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:44.884854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22ad3b-90a7-40ed-9cbc-a4256bcd9512 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:44.889763 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c03083-be85-42e1-8877-166147c05f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:44.890418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c03083-be85-42e1-8877-166147c05f18 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.890702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c03083-be85-42e1-8877-166147c05f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.890980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c03083-be85-42e1-8877-166147c05f18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.891702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c03083-be85-42e1-8877-166147c05f18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:44.893617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c03083-be85-42e1-8877-166147c05f18 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:44.919984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 987/1982] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:44 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:44.924113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04f8b1-f830-403f-b690-d4f51f586278 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:44.935811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04f8b1-f830-403f-b690-d4f51f586278 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:44.991565 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f99bb-149c-43fd-84f6-2628aff591cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:44.992263 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f99bb-149c-43fd-84f6-2628aff591cd admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:44.992557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f99bb-149c-43fd-84f6-2628aff591cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:44.992836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f99bb-149c-43fd-84f6-2628aff591cd admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:44.993611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f99bb-149c-43fd-84f6-2628aff591cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:44.995534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f99bb-149c-43fd-84f6-2628aff591cd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:45.015750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 996/1983] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:44 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:45.019963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c03083-be85-42e1-8877-166147c05f18 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:45.030990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c03083-be85-42e1-8877-166147c05f18 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:45.035622 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116648ee-bb26-461c-971d-4df3996b4fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:45.036312 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116648ee-bb26-461c-971d-4df3996b4fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:45.036601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116648ee-bb26-461c-971d-4df3996b4fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:45.036880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116648ee-bb26-461c-971d-4df3996b4fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:45.037662 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116648ee-bb26-461c-971d-4df3996b4fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:45.039250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116648ee-bb26-461c-971d-4df3996b4fa6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:45.049858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 988/1984] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:45.054091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f99bb-149c-43fd-84f6-2628aff591cd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:45.064353 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f99bb-149c-43fd-84f6-2628aff591cd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:45.069685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b84cdc-9671-4878-b308-3fe818ee6804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:45.070634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b84cdc-9671-4878-b308-3fe818ee6804 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:45.070918 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b84cdc-9671-4878-b308-3fe818ee6804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:45.071197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b84cdc-9671-4878-b308-3fe818ee6804 admin admin] PATH_INFO: `/v3/projects/dda873b396a44871b08a259ef1c9caf6/users/2c34c76e363d434e9087dc901c1bc9ea/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:45.086994 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b84cdc-9671-4878-b308-3fe818ee6804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda873b396a44871b08a259ef1c9caf6, user_id=2c34c76e363d434e9087dc901c1bc9ea, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:45.090183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b84cdc-9671-4878-b308-3fe818ee6804 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:45.106516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 997/1985] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:45 2022] PUT /identity/v3/projects/dda873b396a44871b08a259ef1c9caf6/users/2c34c76e363d434e9087dc901c1bc9ea/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:45.111053 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116648ee-bb26-461c-971d-4df3996b4fa6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:45.122517 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116648ee-bb26-461c-971d-4df3996b4fa6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:45.185529 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c2520-929c-4e10-b9ac-a3264cab0878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:45.186292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c2520-929c-4e10-b9ac-a3264cab0878 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:45.186499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c2520-929c-4e10-b9ac-a3264cab0878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:45.186757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c2520-929c-4e10-b9ac-a3264cab0878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:45.187492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c2520-929c-4e10-b9ac-a3264cab0878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:45.189125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c2520-929c-4e10-b9ac-a3264cab0878 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:45.201458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 989/1986] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:45.205324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b84cdc-9671-4878-b308-3fe818ee6804 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:45.217846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b84cdc-9671-4878-b308-3fe818ee6804 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:45.222583 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2015b-a819-4bf6-87d4-1cb20c51602c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:45.223539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2015b-a819-4bf6-87d4-1cb20c51602c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:45.223937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2015b-a819-4bf6-87d4-1cb20c51602c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:45.224265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2015b-a819-4bf6-87d4-1cb20c51602c admin admin] PATH_INFO: `/v3/projects/dda873b396a44871b08a259ef1c9caf6/users/2c34c76e363d434e9087dc901c1bc9ea/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:45.226384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2015b-a819-4bf6-87d4-1cb20c51602c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda873b396a44871b08a259ef1c9caf6, user_id=2c34c76e363d434e9087dc901c1bc9ea, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:45.229183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2015b-a819-4bf6-87d4-1cb20c51602c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:45.243709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 998/1987] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:45 2022] PUT /identity/v3/projects/dda873b396a44871b08a259ef1c9caf6/users/2c34c76e363d434e9087dc901c1bc9ea/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:45.252087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1eb08c-59fd-4e4d-b17b-684b901f47d5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:45.252736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1eb08c-59fd-4e4d-b17b-684b901f47d5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:45.253114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1eb08c-59fd-4e4d-b17b-684b901f47d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:45.299391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2e1eb08c-59fd-4e4d-b17b-684b901f47d5 None None] MFA Rules not processed for user `2c34c76e363d434e9087dc901c1bc9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:45.300762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2e1eb08c-59fd-4e4d-b17b-684b901f47d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:45.351641 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:45.394091 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 990/1988] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:45.408367 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2015b-a819-4bf6-87d4-1cb20c51602c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:45.410913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-27f2015b-a819-4bf6-87d4-1cb20c51602c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:45.425810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:45.488802 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2015b-a819-4bf6-87d4-1cb20c51602c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:45.493249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee805c2-68f5-4cac-86a7-f57fb1c71c2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:45.493915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee805c2-68f5-4cac-86a7-f57fb1c71c2f service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:45.494204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee805c2-68f5-4cac-86a7-f57fb1c71c2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:45.494483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee805c2-68f5-4cac-86a7-f57fb1c71c2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:45.504472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee805c2-68f5-4cac-86a7-f57fb1c71c2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:45.506361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee805c2-68f5-4cac-86a7-f57fb1c71c2f service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:45.518255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 999/1989] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:45.558022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1eb08c-59fd-4e4d-b17b-684b901f47d5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:45.559411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2e1eb08c-59fd-4e4d-b17b-684b901f47d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:45.571176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1eb08c-59fd-4e4d-b17b-684b901f47d5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:45.575033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cc76f-2cff-474a-b909-5ae2c2a45dc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:45.575853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cc76f-2cff-474a-b909-5ae2c2a45dc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:45.576145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cc76f-2cff-474a-b909-5ae2c2a45dc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:45.576427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cc76f-2cff-474a-b909-5ae2c2a45dc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:45.636695 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cc76f-2cff-474a-b909-5ae2c2a45dc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:45.638138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cc76f-2cff-474a-b909-5ae2c2a45dc0 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:45.649694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 991/1990] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:46.504328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee805c2-68f5-4cac-86a7-f57fb1c71c2f service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:46.506099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fee805c2-68f5-4cac-86a7-f57fb1c71c2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:46.603433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb7f49-1476-4fb2-bb07-d5c387d79e24 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:46.603977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb7f49-1476-4fb2-bb07-d5c387d79e24 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:46.604069 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb7f49-1476-4fb2-bb07-d5c387d79e24 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:46.605394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 992/1991] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 11:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:02:46.622066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8ab80-cb3d-4180-bdf0-5cf3f2f5e0d3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:46.622462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8ab80-cb3d-4180-bdf0-5cf3f2f5e0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:46.622887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8ab80-cb3d-4180-bdf0-5cf3f2f5e0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:46.639293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee805c2-68f5-4cac-86a7-f57fb1c71c2f service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:46.655865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c32f4-c145-4549-9ad5-fd9bdbca56d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:46.657010 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c32f4-c145-4549-9ad5-fd9bdbca56d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:46.657459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c32f4-c145-4549-9ad5-fd9bdbca56d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:46.657889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c32f4-c145-4549-9ad5-fd9bdbca56d1 admin admin] PATH_INFO: `/v3/users/b89dc9b24b2f4890994ea90928856f74` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:46.659382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c32f4-c145-4549-9ad5-fd9bdbca56d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b89dc9b24b2f4890994ea90928856f74)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:46.661382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c32f4-c145-4549-9ad5-fd9bdbca56d1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:46.667438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-15e8ab80-cb3d-4180-bdf0-5cf3f2f5e0d3 None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:46.673022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-15e8ab80-cb3d-4180-bdf0-5cf3f2f5e0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:46.759742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b9c32f4-c145-4549-9ad5-fd9bdbca56d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b89dc9b24b2f4890994ea90928856f74'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:46.760802 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b9c32f4-c145-4549-9ad5-fd9bdbca56d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b89dc9b24b2f4890994ea90928856f74'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:46.761622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b9c32f4-c145-4549-9ad5-fd9bdbca56d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b89dc9b24b2f4890994ea90928856f74'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:46.776507 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b9c32f4-c145-4549-9ad5-fd9bdbca56d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b89dc9b24b2f4890994ea90928856f74'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:46.799575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b9c32f4-c145-4549-9ad5-fd9bdbca56d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b89dc9b24b2f4890994ea90928856f74'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:46.829232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1000/1992] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:46 2022] DELETE /identity/v3/users/b89dc9b24b2f4890994ea90928856f74 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:46.832202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d86991-36f2-4fe0-870c-3d52bf5f6102 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:46.833330 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d86991-36f2-4fe0-870c-3d52bf5f6102 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:46.834191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d86991-36f2-4fe0-870c-3d52bf5f6102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:46.881784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 993/1993] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 11:02:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:46.903685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d0d86991-36f2-4fe0-870c-3d52bf5f6102 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:46.907141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d0d86991-36f2-4fe0-870c-3d52bf5f6102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:46.951353 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:47.007558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8ab80-cb3d-4180-bdf0-5cf3f2f5e0d3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:47.009493 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-15e8ab80-cb3d-4180-bdf0-5cf3f2f5e0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:47.029974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1001/1994] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:47.044279 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8ab80-cb3d-4180-bdf0-5cf3f2f5e0d3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:47.066221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d86991-36f2-4fe0-870c-3d52bf5f6102 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:47.067858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d0d86991-36f2-4fe0-870c-3d52bf5f6102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:47.174452 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:47.187757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cec60-c393-4149-bf0d-925d435026a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:47.188438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cec60-c393-4149-bf0d-925d435026a9 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:47.188720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cec60-c393-4149-bf0d-925d435026a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:47.189125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cec60-c393-4149-bf0d-925d435026a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:47.209145 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d86991-36f2-4fe0-870c-3d52bf5f6102 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:47.214712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95af4b7-64ed-44bb-86f0-5f188d00b047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:47.215379 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95af4b7-64ed-44bb-86f0-5f188d00b047 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:47.215684 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95af4b7-64ed-44bb-86f0-5f188d00b047 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:47.215975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95af4b7-64ed-44bb-86f0-5f188d00b047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:47.226094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95af4b7-64ed-44bb-86f0-5f188d00b047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:47.227049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cec60-c393-4149-bf0d-925d435026a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:47.227555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95af4b7-64ed-44bb-86f0-5f188d00b047 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:47.234645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cec60-c393-4149-bf0d-925d435026a9 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:47.239476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1002/1995] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:47.269720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 994/1996] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:48.087134 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95af4b7-64ed-44bb-86f0-5f188d00b047 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:48.107586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a95af4b7-64ed-44bb-86f0-5f188d00b047 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:48.143536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95af4b7-64ed-44bb-86f0-5f188d00b047 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:48.154521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:48.155501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:48.155882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:48.156234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] PATH_INFO: `/v3/projects/04ae031782034c63885157c3be1b139c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:48.157769 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ae031782034c63885157c3be1b139c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:48.159809 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:48.254550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ae031782034c63885157c3be1b139c', 'request_id': 'req-8247d4d4-05e4-4e11-9081-b37366f1244b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:48.268679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ae031782034c63885157c3be1b139c', 'request_id': 'req-8247d4d4-05e4-4e11-9081-b37366f1244b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:48.284825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ae031782034c63885157c3be1b139c', 'request_id': 'req-8247d4d4-05e4-4e11-9081-b37366f1244b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:48.311000 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] The token cache is being invalidate because project 04ae031782034c63885157c3be1b139c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:48.311611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:48.312834 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:48.317596 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1003/1997] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:48 2022] DELETE /identity/v3/projects/04ae031782034c63885157c3be1b139c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:48.322329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cec60-c393-4149-bf0d-925d435026a9 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:48.340144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cec60-c393-4149-bf0d-925d435026a9 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:48.438853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:48.444143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:48.445062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:48.445510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] PATH_INFO: `/v3/users/65ab2d599264454eac3070d3b7f6e506` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:48.447294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65ab2d599264454eac3070d3b7f6e506)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:48.449665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:48.543049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65ab2d599264454eac3070d3b7f6e506'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:48.543954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65ab2d599264454eac3070d3b7f6e506'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:48.560189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65ab2d599264454eac3070d3b7f6e506'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:48.595591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65ab2d599264454eac3070d3b7f6e506'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:48.625781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65ab2d599264454eac3070d3b7f6e506'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:48.639314 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 995/1998] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:48 2022] DELETE /identity/v3/users/65ab2d599264454eac3070d3b7f6e506 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:48.999434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:49.001751 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:49.026222 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247d4d4-05e4-4e11-9081-b37366f1244b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:49.125294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:49.132324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:49.133623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.134136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.134652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c admin admin] PATH_INFO: `/v3/projects/fc984b18f5dc4c67bb84f0f9f1769564` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.136480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc984b18f5dc4c67bb84f0f9f1769564)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:49.139329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:49.183125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:49.185624 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:49.204092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc984b18f5dc4c67bb84f0f9f1769564', 'request_id': 'req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.212081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc984b18f5dc4c67bb84f0f9f1769564', 'request_id': 'req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.215549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4239f2e-ac82-4334-92d3-80eca68d89e3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:49.223017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc984b18f5dc4c67bb84f0f9f1769564', 'request_id': 'req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.239928 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c admin admin] The token cache is being invalidate because project fc984b18f5dc4c67bb84f0f9f1769564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:49.240140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.241017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0e30ba13-1ec3-4710-a08e-eb70df7dc34c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.247482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1004/1999] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:48 2022] DELETE /identity/v3/projects/fc984b18f5dc4c67bb84f0f9f1769564 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:49.249894 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ef9ea-d7b5-47f1-8feb-05bc9e687983 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.250623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ef9ea-d7b5-47f1-8feb-05bc9e687983 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.251206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ef9ea-d7b5-47f1-8feb-05bc9e687983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.276867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-857ef9ea-d7b5-47f1-8feb-05bc9e687983 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:49.278805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-857ef9ea-d7b5-47f1-8feb-05bc9e687983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:49.288995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4ca06-f28d-4696-86b3-aa4d3f86f2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:49.290711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4ca06-f28d-4696-86b3-aa4d3f86f2d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.290996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4ca06-f28d-4696-86b3-aa4d3f86f2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.291607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4ca06-f28d-4696-86b3-aa4d3f86f2d4 admin admin] PATH_INFO: `/v3/users/13f645dac60041fe9fd6ea9b20d88ff3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.293639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4ca06-f28d-4696-86b3-aa4d3f86f2d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f645dac60041fe9fd6ea9b20d88ff3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:49.295883 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4ca06-f28d-4696-86b3-aa4d3f86f2d4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:49.313893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:49.371227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1005/2000] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:49.378959 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5de4ca06-f28d-4696-86b3-aa4d3f86f2d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f645dac60041fe9fd6ea9b20d88ff3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.379851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5de4ca06-f28d-4696-86b3-aa4d3f86f2d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f645dac60041fe9fd6ea9b20d88ff3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.381166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c73ea-9da2-44dd-b7f1-578b513e8bef None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.381489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c73ea-9da2-44dd-b7f1-578b513e8bef None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.381973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c73ea-9da2-44dd-b7f1-578b513e8bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.388348 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5de4ca06-f28d-4696-86b3-aa4d3f86f2d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f645dac60041fe9fd6ea9b20d88ff3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.403118 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5de4ca06-f28d-4696-86b3-aa4d3f86f2d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f645dac60041fe9fd6ea9b20d88ff3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.412086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5de4ca06-f28d-4696-86b3-aa4d3f86f2d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f645dac60041fe9fd6ea9b20d88ff3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.425328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 996/2001] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:49 2022] DELETE /identity/v3/users/13f645dac60041fe9fd6ea9b20d88ff3 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:49.425524 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-370c73ea-9da2-44dd-b7f1-578b513e8bef None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:49.426107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-370c73ea-9da2-44dd-b7f1-578b513e8bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:49.427469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13482091-6a9d-4744-b0aa-34572660c0dc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.427972 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13482091-6a9d-4744-b0aa-34572660c0dc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.428384 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13482091-6a9d-4744-b0aa-34572660c0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.454596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-13482091-6a9d-4744-b0aa-34572660c0dc None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:49.456949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-13482091-6a9d-4744-b0aa-34572660c0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:49.515860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1006/2002] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:49.516863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c73ea-9da2-44dd-b7f1-578b513e8bef None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:49.528897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c73ea-9da2-44dd-b7f1-578b513e8bef None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:49.535122 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 997/2003] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:49.535787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392dbb3e-a718-4e6f-a575-828a201a2c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:49.536433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392dbb3e-a718-4e6f-a575-828a201a2c84 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.536821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392dbb3e-a718-4e6f-a575-828a201a2c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.537468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392dbb3e-a718-4e6f-a575-828a201a2c84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.540231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392dbb3e-a718-4e6f-a575-828a201a2c84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:49.541198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392dbb3e-a718-4e6f-a575-828a201a2c84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:49.543209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392dbb3e-a718-4e6f-a575-828a201a2c84 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:49.546833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13482091-6a9d-4744-b0aa-34572660c0dc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:49.549612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-13482091-6a9d-4744-b0aa-34572660c0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:49.567148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1007/2004] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:49.571760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392dbb3e-a718-4e6f-a575-828a201a2c84 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:49.589553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392dbb3e-a718-4e6f-a575-828a201a2c84 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:49.595992 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbf51f-f10d-4140-ac93-ecd19da7f6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:49.596595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbf51f-f10d-4140-ac93-ecd19da7f6a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.596883 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbf51f-f10d-4140-ac93-ecd19da7f6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.597163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbf51f-f10d-4140-ac93-ecd19da7f6a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.598537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbf51f-f10d-4140-ac93-ecd19da7f6a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:49.600641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbf51f-f10d-4140-ac93-ecd19da7f6a8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:49.615354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:49.620815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13482091-6a9d-4744-b0aa-34572660c0dc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:49.625583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e734c-7771-492d-a8a3-86d352900c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:49.626542 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e734c-7771-492d-a8a3-86d352900c3d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.626737 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e734c-7771-492d-a8a3-86d352900c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.627086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e734c-7771-492d-a8a3-86d352900c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.633310 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1008/2005] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:49 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:49.637199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbf51f-f10d-4140-ac93-ecd19da7f6a8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:49.637663 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e734c-7771-492d-a8a3-86d352900c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:49.639106 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e734c-7771-492d-a8a3-86d352900c3d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:49.652230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 998/2006] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:49.705456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbf51f-f10d-4140-ac93-ecd19da7f6a8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:49.711351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87fcc0-fb8d-4f7a-973f-47a02c9ef4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:49.712151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87fcc0-fb8d-4f7a-973f-47a02c9ef4ed admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.712817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87fcc0-fb8d-4f7a-973f-47a02c9ef4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.712817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87fcc0-fb8d-4f7a-973f-47a02c9ef4ed admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.713608 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87fcc0-fb8d-4f7a-973f-47a02c9ef4ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:49.715739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87fcc0-fb8d-4f7a-973f-47a02c9ef4ed admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:49.723987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e734c-7771-492d-a8a3-86d352900c3d service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:49.725910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b91e734c-7771-492d-a8a3-86d352900c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:49.740102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e734c-7771-492d-a8a3-86d352900c3d service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:49.743419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1009/2007] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:49 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:49.744631 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87fcc0-fb8d-4f7a-973f-47a02c9ef4ed admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:49.753004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b838fb8-ac83-47da-9b90-f4ce14d4623a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:49.754337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b838fb8-ac83-47da-9b90-f4ce14d4623a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.754584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b838fb8-ac83-47da-9b90-f4ce14d4623a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.754806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b838fb8-ac83-47da-9b90-f4ce14d4623a admin admin] PATH_INFO: `/v3/users/bdb2f7218b2f463a949b401e9862b19a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.756112 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b838fb8-ac83-47da-9b90-f4ce14d4623a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb2f7218b2f463a949b401e9862b19a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:49.759745 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b838fb8-ac83-47da-9b90-f4ce14d4623a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:49.770386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87fcc0-fb8d-4f7a-973f-47a02c9ef4ed admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:49.777782 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787b09f-6043-4110-8a62-4ec5abd93e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:49.778572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787b09f-6043-4110-8a62-4ec5abd93e5e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.779316 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787b09f-6043-4110-8a62-4ec5abd93e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.779316 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787b09f-6043-4110-8a62-4ec5abd93e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.780216 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787b09f-6043-4110-8a62-4ec5abd93e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:49.781968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787b09f-6043-4110-8a62-4ec5abd93e5e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:49.795707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1010/2008] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:49.799342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787b09f-6043-4110-8a62-4ec5abd93e5e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:49.800619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:49.816098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787b09f-6043-4110-8a62-4ec5abd93e5e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:49.826023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5e4f9-184b-481c-ba62-343d336d114b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:49.827203 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5e4f9-184b-481c-ba62-343d336d114b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.828123 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5e4f9-184b-481c-ba62-343d336d114b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.828972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5e4f9-184b-481c-ba62-343d336d114b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.829897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5e4f9-184b-481c-ba62-343d336d114b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:49.832261 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5e4f9-184b-481c-ba62-343d336d114b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:49.834823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7b838fb8-ac83-47da-9b90-f4ce14d4623a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb2f7218b2f463a949b401e9862b19a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.836320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7b838fb8-ac83-47da-9b90-f4ce14d4623a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb2f7218b2f463a949b401e9862b19a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.848353 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1011/2009] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:49.853533 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5e4f9-184b-481c-ba62-343d336d114b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:49.853758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7b838fb8-ac83-47da-9b90-f4ce14d4623a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb2f7218b2f463a949b401e9862b19a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.875455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7b838fb8-ac83-47da-9b90-f4ce14d4623a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb2f7218b2f463a949b401e9862b19a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.876207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7b838fb8-ac83-47da-9b90-f4ce14d4623a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb2f7218b2f463a949b401e9862b19a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:49.877881 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5e4f9-184b-481c-ba62-343d336d114b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:49.883330 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b87ff9-a279-455b-9921-042f20815fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:49.883832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 999/2010] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:49 2022] DELETE /identity/v3/users/bdb2f7218b2f463a949b401e9862b19a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:49.884598 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b87ff9-a279-455b-9921-042f20815fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.884952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b87ff9-a279-455b-9921-042f20815fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.885299 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b87ff9-a279-455b-9921-042f20815fcc admin admin] PATH_INFO: `/v3/projects/4b7f8a3cc4db4355ae2093b4b9b5199d/users/f2b2bb8167764cf48f42d0b466a1b6b5/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.887941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9f633-aab2-40da-9c16-f46eb5cfbcfa None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.888464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9f633-aab2-40da-9c16-f46eb5cfbcfa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.888848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9f633-aab2-40da-9c16-f46eb5cfbcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.902841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b87ff9-a279-455b-9921-042f20815fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7f8a3cc4db4355ae2093b4b9b5199d, user_id=f2b2bb8167764cf48f42d0b466a1b6b5, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:49.906770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b87ff9-a279-455b-9921-042f20815fcc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:49.912838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d4b9f633-aab2-40da-9c16-f46eb5cfbcfa None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:49.914330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d4b9f633-aab2-40da-9c16-f46eb5cfbcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:49.931925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1012/2011] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:49 2022] PUT /identity/v3/projects/4b7f8a3cc4db4355ae2093b4b9b5199d/users/f2b2bb8167764cf48f42d0b466a1b6b5/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:49.946225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddf7c5-19be-430d-8c6c-5bf14036d587 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:49.946539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddf7c5-19be-430d-8c6c-5bf14036d587 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:49.946828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddf7c5-19be-430d-8c6c-5bf14036d587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:49.988584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d2ddf7c5-19be-430d-8c6c-5bf14036d587 None None] MFA Rules not processed for user `f2b2bb8167764cf48f42d0b466a1b6b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:49.990138 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d2ddf7c5-19be-430d-8c6c-5bf14036d587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:50.023713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1000/2012] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:50.037803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9f633-aab2-40da-9c16-f46eb5cfbcfa None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:50.039729 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d4b9f633-aab2-40da-9c16-f46eb5cfbcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:50.077794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1013/2013] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:50.091453 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddf7c5-19be-430d-8c6c-5bf14036d587 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:50.093191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d2ddf7c5-19be-430d-8c6c-5bf14036d587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:50.109814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9f633-aab2-40da-9c16-f46eb5cfbcfa None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:50.138308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:50.177075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddf7c5-19be-430d-8c6c-5bf14036d587 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:50.182909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ae21b-5878-483f-b291-6b805f5e7726 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af6cc8c56db48beae2550954446c119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:50.183980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ae21b-5878-483f-b291-6b805f5e7726 service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:50.184364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ae21b-5878-483f-b291-6b805f5e7726 service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:50.184828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ae21b-5878-483f-b291-6b805f5e7726 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:50.189892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cc3c0-d97b-4c22-9eac-012c6dd34b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:50.190559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cc3c0-d97b-4c22-9eac-012c6dd34b98 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:50.190844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cc3c0-d97b-4c22-9eac-012c6dd34b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:50.191122 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cc3c0-d97b-4c22-9eac-012c6dd34b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:50.196050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ae21b-5878-483f-b291-6b805f5e7726 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:50.197511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ae21b-5878-483f-b291-6b805f5e7726 service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:50.205675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:50.207037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1014/2014] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:50 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:50.209216 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ae21b-5878-483f-b291-6b805f5e7726 service glance] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:50.210945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f28ae21b-5878-483f-b291-6b805f5e7726 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:50.273996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5cc3c0-d97b-4c22-9eac-012c6dd34b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:50.275574 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5cc3c0-d97b-4c22-9eac-012c6dd34b98 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:50.286237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1001/2015] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:50.297839 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ae21b-5878-483f-b291-6b805f5e7726 service glance] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:50.305838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:50.306715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:50.307422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:50.311800 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] PATH_INFO: `/v3/projects/264d23c899ae4b3289da270d559c1759` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:50.313395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] RBAC: Authorizing `identity:delete_project(project_id=264d23c899ae4b3289da270d559c1759)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:50.318270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:50.420493 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '264d23c899ae4b3289da270d559c1759', 'request_id': 'req-944dd98c-8eca-45d9-88cc-3bfc913feebc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:50.432368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '264d23c899ae4b3289da270d559c1759', 'request_id': 'req-944dd98c-8eca-45d9-88cc-3bfc913feebc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:50.449764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '264d23c899ae4b3289da270d559c1759', 'request_id': 'req-944dd98c-8eca-45d9-88cc-3bfc913feebc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:50.484384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] The token cache is being invalidate because project 264d23c899ae4b3289da270d559c1759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:50.485394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:50.486873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:50.499067 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1015/2016] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:50 2022] DELETE /identity/v3/projects/264d23c899ae4b3289da270d559c1759 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:50.500674 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cc3c0-d97b-4c22-9eac-012c6dd34b98 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:50.529172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cc3c0-d97b-4c22-9eac-012c6dd34b98 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:50.652955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:50.654339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:50.654467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:50.654814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] PATH_INFO: `/v3/users/15fa41a31d6a434eae4210de5161a9d3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:50.657115 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15fa41a31d6a434eae4210de5161a9d3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:50.662113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:50.783870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15fa41a31d6a434eae4210de5161a9d3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:50.784555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15fa41a31d6a434eae4210de5161a9d3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:50.801215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15fa41a31d6a434eae4210de5161a9d3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:50.826161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15fa41a31d6a434eae4210de5161a9d3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:50.840228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15fa41a31d6a434eae4210de5161a9d3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:50.849050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1002/2017] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:50 2022] DELETE /identity/v3/users/15fa41a31d6a434eae4210de5161a9d3 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:51.366329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:51.368858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:51.390833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dd98c-8eca-45d9-88cc-3bfc913feebc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:51.392942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:51.394477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:51.468659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:51.498826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:51.500025 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:51.500435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:51.500814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b admin admin] PATH_INFO: `/v3/projects/34d70fd94675477e9e0a1971d9454a7a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:51.503201 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d70fd94675477e9e0a1971d9454a7a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:51.505683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:51.507747 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:51.518714 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266406bc-4a6b-4495-a5f8-b3955701ae39 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:51.546119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:51.547016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:51.547317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:51.547565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] PATH_INFO: `/v3/projects/f61e05a59f2b41f6bca7aed76ac258a0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:51.550833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f61e05a59f2b41f6bca7aed76ac258a0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:51.554210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:51.588266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d70fd94675477e9e0a1971d9454a7a', 'request_id': 'req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.609282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d70fd94675477e9e0a1971d9454a7a', 'request_id': 'req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.620356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d70fd94675477e9e0a1971d9454a7a', 'request_id': 'req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.639508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f61e05a59f2b41f6bca7aed76ac258a0', 'request_id': 'req-074bd11c-56fe-4fff-a80b-0f81e62e88e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.641367 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b admin admin] The token cache is being invalidate because project 34d70fd94675477e9e0a1971d9454a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:51.642020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.643688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec8eba98-0dad-4e6c-80a0-81d1dd44943b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.648484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1016/2018] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:51 2022] DELETE /identity/v3/projects/34d70fd94675477e9e0a1971d9454a7a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:51.652445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f61e05a59f2b41f6bca7aed76ac258a0', 'request_id': 'req-074bd11c-56fe-4fff-a80b-0f81e62e88e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.657436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcb903-efad-452c-848a-dd87ffcfe509 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:51.657951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcb903-efad-452c-848a-dd87ffcfe509 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:51.658405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcb903-efad-452c-848a-dd87ffcfe509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:51.661448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f61e05a59f2b41f6bca7aed76ac258a0', 'request_id': 'req-074bd11c-56fe-4fff-a80b-0f81e62e88e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.689068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] The token cache is being invalidate because project f61e05a59f2b41f6bca7aed76ac258a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:51.689068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.689068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.689068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1003/2019] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:51 2022] DELETE /identity/v3/projects/f61e05a59f2b41f6bca7aed76ac258a0 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:51.691449 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-63bcb903-efad-452c-848a-dd87ffcfe509 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:51.692691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-63bcb903-efad-452c-848a-dd87ffcfe509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:51.694901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:51.696372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:51.794620 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bd11c-56fe-4fff-a80b-0f81e62e88e4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:51.802448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:51.803681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:51.804100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:51.804488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] PATH_INFO: `/v3/users/2c34c76e363d434e9087dc901c1bc9ea` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:51.806754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c34c76e363d434e9087dc901c1bc9ea)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:51.809633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:51.841197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1017/2020] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:51.842741 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68119cbc-902c-4703-b4b1-cd47786acf48 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:51.843181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68119cbc-902c-4703-b4b1-cd47786acf48 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:51.843467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68119cbc-902c-4703-b4b1-cd47786acf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:51.886614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-68119cbc-902c-4703-b4b1-cd47786acf48 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:51.887887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-68119cbc-902c-4703-b4b1-cd47786acf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:51.896559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c34c76e363d434e9087dc901c1bc9ea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.897859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c34c76e363d434e9087dc901c1bc9ea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.903033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1018/2021] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:51.904155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68119cbc-902c-4703-b4b1-cd47786acf48 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:51.914600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c34c76e363d434e9087dc901c1bc9ea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.923896 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68119cbc-902c-4703-b4b1-cd47786acf48 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:51.925508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:51.933891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd89136-bb24-4997-928a-f9a6618ba22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:51.934686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd89136-bb24-4997-928a-f9a6618ba22a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:51.935056 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd89136-bb24-4997-928a-f9a6618ba22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:51.935431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd89136-bb24-4997-928a-f9a6618ba22a admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:51.936313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd89136-bb24-4997-928a-f9a6618ba22a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:51.936683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd89136-bb24-4997-928a-f9a6618ba22a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:51.938110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c34c76e363d434e9087dc901c1bc9ea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.939984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd89136-bb24-4997-928a-f9a6618ba22a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:51.950520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c34c76e363d434e9087dc901c1bc9ea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:51.970058 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1004/2022] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:51 2022] DELETE /identity/v3/users/2c34c76e363d434e9087dc901c1bc9ea => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:51.978928 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1019/2023] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:51.984260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:51.993095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:52.158119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28114b-7f0b-4c49-a382-d16fde4e1108 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:52.163966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817fdf8-9662-42a9-aa8e-4002c6df9d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:52.177866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817fdf8-9662-42a9-aa8e-4002c6df9d19 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:52.178170 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817fdf8-9662-42a9-aa8e-4002c6df9d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:52.179095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817fdf8-9662-42a9-aa8e-4002c6df9d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:52.186522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817fdf8-9662-42a9-aa8e-4002c6df9d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:52.190337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817fdf8-9662-42a9-aa8e-4002c6df9d19 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:52.263464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1005/2024] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:51 2022] POST /identity/v3/projects => generated 359 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:52.267363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd89136-bb24-4997-928a-f9a6618ba22a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:52.280712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd89136-bb24-4997-928a-f9a6618ba22a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:52.422522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817fdf8-9662-42a9-aa8e-4002c6df9d19 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:52.424585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4817fdf8-9662-42a9-aa8e-4002c6df9d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:52.445208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5f9cc-045d-4ccd-8d77-6816f003c5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:52.447069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5f9cc-045d-4ccd-8d77-6816f003c5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:52.448209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5f9cc-045d-4ccd-8d77-6816f003c5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:52.448998 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5f9cc-045d-4ccd-8d77-6816f003c5c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:52.453371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5f9cc-045d-4ccd-8d77-6816f003c5c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:52.456043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5f9cc-045d-4ccd-8d77-6816f003c5c5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:52.490424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:52.519067 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1020/2025] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:52 2022] POST /identity/v3/users => generated 280 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:52.520196 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5f9cc-045d-4ccd-8d77-6816f003c5c5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:52.537565 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5f9cc-045d-4ccd-8d77-6816f003c5c5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:52.543915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278c10f-6c25-4c16-b8ff-fdafaf63937e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:52.544922 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278c10f-6c25-4c16-b8ff-fdafaf63937e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:52.545403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278c10f-6c25-4c16-b8ff-fdafaf63937e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:52.545962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278c10f-6c25-4c16-b8ff-fdafaf63937e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:52.547256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278c10f-6c25-4c16-b8ff-fdafaf63937e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:52.549148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278c10f-6c25-4c16-b8ff-fdafaf63937e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:52.570804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1021/2026] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:52.574921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278c10f-6c25-4c16-b8ff-fdafaf63937e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:52.578464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817fdf8-9662-42a9-aa8e-4002c6df9d19 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:52.588505 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278c10f-6c25-4c16-b8ff-fdafaf63937e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:52.592106 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc1731-dfce-4b5e-81bd-840276cca2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:52.593856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc1731-dfce-4b5e-81bd-840276cca2ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:52.594300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc1731-dfce-4b5e-81bd-840276cca2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:52.594857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc1731-dfce-4b5e-81bd-840276cca2ad admin admin] PATH_INFO: `/v3/projects/dda873b396a44871b08a259ef1c9caf6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:52.596595 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc1731-dfce-4b5e-81bd-840276cca2ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda873b396a44871b08a259ef1c9caf6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:52.598711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc1731-dfce-4b5e-81bd-840276cca2ad admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:52.602323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138d92d-10fb-4b79-a314-7562b357256b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:52.603176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138d92d-10fb-4b79-a314-7562b357256b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:52.603643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138d92d-10fb-4b79-a314-7562b357256b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:52.606454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138d92d-10fb-4b79-a314-7562b357256b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:52.608497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138d92d-10fb-4b79-a314-7562b357256b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:52.610334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138d92d-10fb-4b79-a314-7562b357256b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:52.636642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1022/2027] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:52.637813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138d92d-10fb-4b79-a314-7562b357256b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:52.659804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6cfc1731-dfce-4b5e-81bd-840276cca2ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda873b396a44871b08a259ef1c9caf6', 'request_id': 'req-6cfc1731-dfce-4b5e-81bd-840276cca2ad'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:52.664354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138d92d-10fb-4b79-a314-7562b357256b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:52.678385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6cfc1731-dfce-4b5e-81bd-840276cca2ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda873b396a44871b08a259ef1c9caf6', 'request_id': 'req-6cfc1731-dfce-4b5e-81bd-840276cca2ad'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:52.691231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6cfc1731-dfce-4b5e-81bd-840276cca2ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda873b396a44871b08a259ef1c9caf6', 'request_id': 'req-6cfc1731-dfce-4b5e-81bd-840276cca2ad'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:52.692796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:52.710830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6cfc1731-dfce-4b5e-81bd-840276cca2ad admin admin] The token cache is being invalidate because project dda873b396a44871b08a259ef1c9caf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:52.711283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6cfc1731-dfce-4b5e-81bd-840276cca2ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:52.714772 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6cfc1731-dfce-4b5e-81bd-840276cca2ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:52.723828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1006/2028] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:52 2022] DELETE /identity/v3/projects/dda873b396a44871b08a259ef1c9caf6 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:52.727397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764213d-c8ed-4a04-b461-fb19afdba223 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:52.727873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764213d-c8ed-4a04-b461-fb19afdba223 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:52.728441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764213d-c8ed-4a04-b461-fb19afdba223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:52.772428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c764213d-c8ed-4a04-b461-fb19afdba223 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:52.777140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c764213d-c8ed-4a04-b461-fb19afdba223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:52.777246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b31243-c7fb-4713-bb7a-184a1a914e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:52.778223 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b31243-c7fb-4713-bb7a-184a1a914e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:52.778588 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b31243-c7fb-4713-bb7a-184a1a914e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:52.778874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b31243-c7fb-4713-bb7a-184a1a914e6a admin admin] PATH_INFO: `/v3/projects/16ce84ce8d0f4c7fb261b6ccda428fb0/users/a52c7d8087194425ac1767b74b740f70/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:52.806295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1007/2029] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:52.811910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b6852-3220-48de-8a95-1a26333dcc85 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:52.812361 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b6852-3220-48de-8a95-1a26333dcc85 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:52.812744 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b6852-3220-48de-8a95-1a26333dcc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:52.814035 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b31243-c7fb-4713-bb7a-184a1a914e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=16ce84ce8d0f4c7fb261b6ccda428fb0, user_id=a52c7d8087194425ac1767b74b740f70, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:52.818457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b31243-c7fb-4713-bb7a-184a1a914e6a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:52.853180 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1023/2030] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:52 2022] PUT /identity/v3/projects/16ce84ce8d0f4c7fb261b6ccda428fb0/users/a52c7d8087194425ac1767b74b740f70/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:52.866842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af6086-d56f-45d5-8b47-5f486eea4ab1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:52.867169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af6086-d56f-45d5-8b47-5f486eea4ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:52.867458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af6086-d56f-45d5-8b47-5f486eea4ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:52.870543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-851b6852-3220-48de-8a95-1a26333dcc85 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:52.872735 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-851b6852-3220-48de-8a95-1a26333dcc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:52.931818 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-73af6086-d56f-45d5-8b47-5f486eea4ab1 None None] MFA Rules not processed for user `a52c7d8087194425ac1767b74b740f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:52.933125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-73af6086-d56f-45d5-8b47-5f486eea4ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:53.020465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1008/2031] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:53.021875 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b6852-3220-48de-8a95-1a26333dcc85 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:53.036233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b6852-3220-48de-8a95-1a26333dcc85 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:53.042027 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2f571-f8d4-4b50-ac26-f6eadda788bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:53.042782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2f571-f8d4-4b50-ac26-f6eadda788bc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.043073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2f571-f8d4-4b50-ac26-f6eadda788bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.043404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2f571-f8d4-4b50-ac26-f6eadda788bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.044291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2f571-f8d4-4b50-ac26-f6eadda788bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:53.044609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2f571-f8d4-4b50-ac26-f6eadda788bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:53.046628 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2f571-f8d4-4b50-ac26-f6eadda788bc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:53.063368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1009/2032] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:02:53.066210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1024/2033] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:52 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:53.067433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2f571-f8d4-4b50-ac26-f6eadda788bc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:53.080658 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2f571-f8d4-4b50-ac26-f6eadda788bc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:53.104364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5b4b4-2949-401c-bd9b-ce1fd13a5a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:53.105227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5b4b4-2949-401c-bd9b-ce1fd13a5a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.105522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5b4b4-2949-401c-bd9b-ce1fd13a5a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.105805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5b4b4-2949-401c-bd9b-ce1fd13a5a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.106517 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5b4b4-2949-401c-bd9b-ce1fd13a5a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:53.106633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af6086-d56f-45d5-8b47-5f486eea4ab1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:53.108027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-73af6086-d56f-45d5-8b47-5f486eea4ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:53.108479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5b4b4-2949-401c-bd9b-ce1fd13a5a13 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:53.158457 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1010/2034] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:53 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:53.162709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5b4b4-2949-401c-bd9b-ce1fd13a5a13 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:53.207200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af6086-d56f-45d5-8b47-5f486eea4ab1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:53.212920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:53.238627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5b4b4-2949-401c-bd9b-ce1fd13a5a13 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:53.247273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3c0de-e711-4abd-ab3c-3457ba5e7fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:53.248732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3c0de-e711-4abd-ab3c-3457ba5e7fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.248732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3c0de-e711-4abd-ab3c-3457ba5e7fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.248732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3c0de-e711-4abd-ab3c-3457ba5e7fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.249235 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3c0de-e711-4abd-ab3c-3457ba5e7fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:53.251317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3c0de-e711-4abd-ab3c-3457ba5e7fb3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:53.273185 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c04025-1bf4-4866-90e3-e3547cc3c780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:53.273967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c04025-1bf4-4866-90e3-e3547cc3c780 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.274319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c04025-1bf4-4866-90e3-e3547cc3c780 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.274669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c04025-1bf4-4866-90e3-e3547cc3c780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.278781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1011/2035] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:53 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:53.284882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3c0de-e711-4abd-ab3c-3457ba5e7fb3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:53.298991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3c0de-e711-4abd-ab3c-3457ba5e7fb3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:53.304111 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149a373-043b-49bd-a248-1fe083d73d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:53.305628 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149a373-043b-49bd-a248-1fe083d73d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.305938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149a373-043b-49bd-a248-1fe083d73d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.306222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149a373-043b-49bd-a248-1fe083d73d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.306960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149a373-043b-49bd-a248-1fe083d73d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:53.309358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149a373-043b-49bd-a248-1fe083d73d1b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:53.322476 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1012/2036] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:53.326180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149a373-043b-49bd-a248-1fe083d73d1b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:53.337174 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149a373-043b-49bd-a248-1fe083d73d1b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:53.342187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52259c4-d149-4894-ae25-92a5b721c9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:53.342893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52259c4-d149-4894-ae25-92a5b721c9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.343182 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52259c4-d149-4894-ae25-92a5b721c9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.343467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52259c4-d149-4894-ae25-92a5b721c9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.344288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52259c4-d149-4894-ae25-92a5b721c9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:53.345868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52259c4-d149-4894-ae25-92a5b721c9b6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:53.347635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c04025-1bf4-4866-90e3-e3547cc3c780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:53.349456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c04025-1bf4-4866-90e3-e3547cc3c780 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:53.359442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1025/2037] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:53 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:53.367813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1013/2038] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:02:53.370794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c04025-1bf4-4866-90e3-e3547cc3c780 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:53.376682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52259c4-d149-4894-ae25-92a5b721c9b6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:53.378400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e52259c4-d149-4894-ae25-92a5b721c9b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:53.389237 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c04025-1bf4-4866-90e3-e3547cc3c780 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:53.392522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52259c4-d149-4894-ae25-92a5b721c9b6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:53.395350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ff334-d474-47cb-aee0-5e1a105f12cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:53.396707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ff334-d474-47cb-aee0-5e1a105f12cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.396999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ff334-d474-47cb-aee0-5e1a105f12cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.397975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ff334-d474-47cb-aee0-5e1a105f12cc admin admin] PATH_INFO: `/v3/projects/64682ed401c74af7aa436a2171819ebe/users/1b0423a12e39453e9cd6d1e8ab43af60/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.398100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15da62-2f3a-4795-a521-06b1d17bf204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:53.398850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15da62-2f3a-4795-a521-06b1d17bf204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.399132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15da62-2f3a-4795-a521-06b1d17bf204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.399406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15da62-2f3a-4795-a521-06b1d17bf204 admin admin] PATH_INFO: `/v3/users/a52c7d8087194425ac1767b74b740f70` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.400796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15da62-2f3a-4795-a521-06b1d17bf204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a52c7d8087194425ac1767b74b740f70)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:53.402729 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15da62-2f3a-4795-a521-06b1d17bf204 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:53.414666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ff334-d474-47cb-aee0-5e1a105f12cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=64682ed401c74af7aa436a2171819ebe, user_id=1b0423a12e39453e9cd6d1e8ab43af60, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:53.417822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ff334-d474-47cb-aee0-5e1a105f12cc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:53.419364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:53.434904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1026/2039] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:53 2022] PUT /identity/v3/projects/64682ed401c74af7aa436a2171819ebe/users/1b0423a12e39453e9cd6d1e8ab43af60/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:53.440396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9dac2-d26b-488b-928c-9e421006ac66 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.440687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9dac2-d26b-488b-928c-9e421006ac66 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.440969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9dac2-d26b-488b-928c-9e421006ac66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.462505 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dec9dac2-d26b-488b-928c-9e421006ac66 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:53.463734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dec9dac2-d26b-488b-928c-9e421006ac66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:53.476075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4c15da62-2f3a-4795-a521-06b1d17bf204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a52c7d8087194425ac1767b74b740f70'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:53.477293 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4c15da62-2f3a-4795-a521-06b1d17bf204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a52c7d8087194425ac1767b74b740f70'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:53.499296 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4c15da62-2f3a-4795-a521-06b1d17bf204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a52c7d8087194425ac1767b74b740f70'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:53.515746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4c15da62-2f3a-4795-a521-06b1d17bf204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a52c7d8087194425ac1767b74b740f70'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:53.523774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4c15da62-2f3a-4795-a521-06b1d17bf204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a52c7d8087194425ac1767b74b740f70'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:53.533695 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1014/2040] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:53 2022] DELETE /identity/v3/users/a52c7d8087194425ac1767b74b740f70 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:53.535478 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b40e3-72ac-4b7d-afad-89b89a3be0c6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.536216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b40e3-72ac-4b7d-afad-89b89a3be0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.536731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b40e3-72ac-4b7d-afad-89b89a3be0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.545889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1027/2041] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:53.557347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cf8b40e3-72ac-4b7d-afad-89b89a3be0c6 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:53.558816 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cf8b40e3-72ac-4b7d-afad-89b89a3be0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:53.565066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9dac2-d26b-488b-928c-9e421006ac66 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:53.566290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dec9dac2-d26b-488b-928c-9e421006ac66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:53.624207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:53.642204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1015/2042] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:53.650962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9dac2-d26b-488b-928c-9e421006ac66 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:53.658905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b40e3-72ac-4b7d-afad-89b89a3be0c6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:53.660644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cf8b40e3-72ac-4b7d-afad-89b89a3be0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:53.743327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0953dd3-e4a0-4902-a981-2e4ddd8e7b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:53.745700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0953dd3-e4a0-4902-a981-2e4ddd8e7b79 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.746113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0953dd3-e4a0-4902-a981-2e4ddd8e7b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.748090 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0953dd3-e4a0-4902-a981-2e4ddd8e7b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.776035 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0953dd3-e4a0-4902-a981-2e4ddd8e7b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:53.779097 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0953dd3-e4a0-4902-a981-2e4ddd8e7b79 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:53.786858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b40e3-72ac-4b7d-afad-89b89a3be0c6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:53.791835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e2c49-b152-44a1-b3f4-2f02206b35ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:53.792724 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e2c49-b152-44a1-b3f4-2f02206b35ea service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.793070 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e2c49-b152-44a1-b3f4-2f02206b35ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.794259 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e2c49-b152-44a1-b3f4-2f02206b35ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.806250 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1028/2043] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:53.809504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e2c49-b152-44a1-b3f4-2f02206b35ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:53.811199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e2c49-b152-44a1-b3f4-2f02206b35ea service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:53.828382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1016/2044] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:55.187942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0953dd3-e4a0-4902-a981-2e4ddd8e7b79 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:55.190175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b0953dd3-e4a0-4902-a981-2e4ddd8e7b79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:55.215888 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0953dd3-e4a0-4902-a981-2e4ddd8e7b79 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:55.228310 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763542fa-13d5-4328-a29b-e40d1196dbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:55.229473 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763542fa-13d5-4328-a29b-e40d1196dbe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:55.231803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763542fa-13d5-4328-a29b-e40d1196dbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:55.232460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763542fa-13d5-4328-a29b-e40d1196dbe2 admin admin] PATH_INFO: `/v3/projects/16ce84ce8d0f4c7fb261b6ccda428fb0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:55.238021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763542fa-13d5-4328-a29b-e40d1196dbe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16ce84ce8d0f4c7fb261b6ccda428fb0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:55.242614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763542fa-13d5-4328-a29b-e40d1196dbe2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:55.263567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afa7a2-8ade-48ba-9c85-d5867783c40e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:55.264149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afa7a2-8ade-48ba-9c85-d5867783c40e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:55.264149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afa7a2-8ade-48ba-9c85-d5867783c40e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:55.265741 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1017/2045] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 11:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:55.273111 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e13169-f67a-4ff9-867c-0b8f93eed3bf None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:55.273402 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e13169-f67a-4ff9-867c-0b8f93eed3bf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:55.273685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e13169-f67a-4ff9-867c-0b8f93eed3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:55.330433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-03e13169-f67a-4ff9-867c-0b8f93eed3bf None None] MFA Rules not processed for user `18fc0e7956b14a49beab025922e8ff81`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:55.334105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-03e13169-f67a-4ff9-867c-0b8f93eed3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:55.342228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-763542fa-13d5-4328-a29b-e40d1196dbe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16ce84ce8d0f4c7fb261b6ccda428fb0', 'request_id': 'req-763542fa-13d5-4328-a29b-e40d1196dbe2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:55.352177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-763542fa-13d5-4328-a29b-e40d1196dbe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16ce84ce8d0f4c7fb261b6ccda428fb0', 'request_id': 'req-763542fa-13d5-4328-a29b-e40d1196dbe2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:55.370982 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-763542fa-13d5-4328-a29b-e40d1196dbe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16ce84ce8d0f4c7fb261b6ccda428fb0', 'request_id': 'req-763542fa-13d5-4328-a29b-e40d1196dbe2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:55.399924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:55.400553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-763542fa-13d5-4328-a29b-e40d1196dbe2 admin admin] The token cache is being invalidate because project 16ce84ce8d0f4c7fb261b6ccda428fb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:55.400927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-763542fa-13d5-4328-a29b-e40d1196dbe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:55.402114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-763542fa-13d5-4328-a29b-e40d1196dbe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:55.404203 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1029/2046] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:55 2022] DELETE /identity/v3/projects/16ce84ce8d0f4c7fb261b6ccda428fb0 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:55.409908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691ddd0-afa3-42c7-afd9-06fcffa64d5e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:55.410202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691ddd0-afa3-42c7-afd9-06fcffa64d5e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:55.410489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691ddd0-afa3-42c7-afd9-06fcffa64d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:55.436452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8691ddd0-afa3-42c7-afd9-06fcffa64d5e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:55.437724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8691ddd0-afa3-42c7-afd9-06fcffa64d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:55.449102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1018/2047] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 11:02:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:55.461499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e13169-f67a-4ff9-867c-0b8f93eed3bf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:55.462940 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-03e13169-f67a-4ff9-867c-0b8f93eed3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:55.477779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e13169-f67a-4ff9-867c-0b8f93eed3bf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:55.483169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44ac21-0fcc-4f5b-9bb6-6503dd318243 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:55.484308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44ac21-0fcc-4f5b-9bb6-6503dd318243 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:55.484308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44ac21-0fcc-4f5b-9bb6-6503dd318243 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:55.484495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44ac21-0fcc-4f5b-9bb6-6503dd318243 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:55.495908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44ac21-0fcc-4f5b-9bb6-6503dd318243 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:55.498174 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44ac21-0fcc-4f5b-9bb6-6503dd318243 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:55.509631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1019/2048] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:02:55.554916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1030/2049] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:55.563821 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bc2fe-9e0f-4b50-a214-c7da90cd20f7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:55.564116 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bc2fe-9e0f-4b50-a214-c7da90cd20f7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:55.564399 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bc2fe-9e0f-4b50-a214-c7da90cd20f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:55.583095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-799bc2fe-9e0f-4b50-a214-c7da90cd20f7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:55.584976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-799bc2fe-9e0f-4b50-a214-c7da90cd20f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:55.593292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1020/2050] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:55.597524 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691ddd0-afa3-42c7-afd9-06fcffa64d5e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:55.608045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691ddd0-afa3-42c7-afd9-06fcffa64d5e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:55.612827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84706d7a-bbef-4349-9e35-b6bcd67b37ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:55.613486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84706d7a-bbef-4349-9e35-b6bcd67b37ea admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:55.613773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84706d7a-bbef-4349-9e35-b6bcd67b37ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:55.614055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84706d7a-bbef-4349-9e35-b6bcd67b37ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:55.614787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84706d7a-bbef-4349-9e35-b6bcd67b37ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:55.615098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84706d7a-bbef-4349-9e35-b6bcd67b37ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:55.617323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84706d7a-bbef-4349-9e35-b6bcd67b37ea admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:55.631697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1031/2051] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:02:55.634932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bc2fe-9e0f-4b50-a214-c7da90cd20f7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:55.645135 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bc2fe-9e0f-4b50-a214-c7da90cd20f7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:55.649784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488de960-832e-4b01-b457-9b8846946e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:55.651738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488de960-832e-4b01-b457-9b8846946e91 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:55.651738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488de960-832e-4b01-b457-9b8846946e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:55.651994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488de960-832e-4b01-b457-9b8846946e91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:55.652707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488de960-832e-4b01-b457-9b8846946e91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:55.654609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488de960-832e-4b01-b457-9b8846946e91 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:55.684079 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1021/2052] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:55 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:55.688197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84706d7a-bbef-4349-9e35-b6bcd67b37ea admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:55.698844 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84706d7a-bbef-4349-9e35-b6bcd67b37ea admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:55.703506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:55.796397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda91a5-a3a8-45cd-a2ca-91c7a3d12126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:55.799348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda91a5-a3a8-45cd-a2ca-91c7a3d12126 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:55.800009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda91a5-a3a8-45cd-a2ca-91c7a3d12126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:55.800442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda91a5-a3a8-45cd-a2ca-91c7a3d12126 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:55.825304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda91a5-a3a8-45cd-a2ca-91c7a3d12126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:55.831638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda91a5-a3a8-45cd-a2ca-91c7a3d12126 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:55.841090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ab2db-0d89-4480-a2d8-47aeee445370 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:55.841627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ab2db-0d89-4480-a2d8-47aeee445370 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:55.842086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ab2db-0d89-4480-a2d8-47aeee445370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:55.916352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1032/2053] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:55 2022] POST /identity/v3/users => generated 288 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:02:55.917263 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda91a5-a3a8-45cd-a2ca-91c7a3d12126 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:55.937272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1d1ab2db-0d89-4480-a2d8-47aeee445370 None None] MFA Rules not processed for user `1b0423a12e39453e9cd6d1e8ab43af60`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:55.939149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1d1ab2db-0d89-4480-a2d8-47aeee445370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:55.950113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda91a5-a3a8-45cd-a2ca-91c7a3d12126 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:55.963990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931d03b-7021-419d-9809-09bdb1e0eee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:55.964342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931d03b-7021-419d-9809-09bdb1e0eee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:55.964626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931d03b-7021-419d-9809-09bdb1e0eee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:55.966262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931d03b-7021-419d-9809-09bdb1e0eee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:55.968592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931d03b-7021-419d-9809-09bdb1e0eee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:55.970252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931d03b-7021-419d-9809-09bdb1e0eee3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:56.014845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1033/2054] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:56.020708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931d03b-7021-419d-9809-09bdb1e0eee3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:56.037094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931d03b-7021-419d-9809-09bdb1e0eee3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:56.068080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a00f9-3735-4f26-886e-bc1e3e451b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:56.068586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a00f9-3735-4f26-886e-bc1e3e451b97 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:56.069008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a00f9-3735-4f26-886e-bc1e3e451b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:56.071140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a00f9-3735-4f26-886e-bc1e3e451b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:56.071948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a00f9-3735-4f26-886e-bc1e3e451b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:56.077367 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a00f9-3735-4f26-886e-bc1e3e451b97 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:56.107762 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1034/2055] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:56.109968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a00f9-3735-4f26-886e-bc1e3e451b97 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:56.116889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1022/2056] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:56.123468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a00f9-3735-4f26-886e-bc1e3e451b97 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:56.126126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ab2db-0d89-4480-a2d8-47aeee445370 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:56.127588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1d1ab2db-0d89-4480-a2d8-47aeee445370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:56.128523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c67c7-07e1-4bbc-b83a-f94e1eeb5d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:56.129515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c67c7-07e1-4bbc-b83a-f94e1eeb5d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:56.129802 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c67c7-07e1-4bbc-b83a-f94e1eeb5d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:56.130176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c67c7-07e1-4bbc-b83a-f94e1eeb5d0e admin admin] PATH_INFO: `/v3/projects/218e8654a93f4ff1b3125c91f8fc892c/users/9337c184fac44c6c815f40f5059f7056/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:56.149019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c67c7-07e1-4bbc-b83a-f94e1eeb5d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=218e8654a93f4ff1b3125c91f8fc892c, user_id=9337c184fac44c6c815f40f5059f7056, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:56.152007 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c67c7-07e1-4bbc-b83a-f94e1eeb5d0e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:56.170231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1035/2057] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:56 2022] PUT /identity/v3/projects/218e8654a93f4ff1b3125c91f8fc892c/users/9337c184fac44c6c815f40f5059f7056/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:56.181507 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c98a2-7f7d-4012-816d-73695583d2d2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:56.181850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c98a2-7f7d-4012-816d-73695583d2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:56.182174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c98a2-7f7d-4012-816d-73695583d2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:56.223300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2c8c98a2-7f7d-4012-816d-73695583d2d2 None None] MFA Rules not processed for user `9337c184fac44c6c815f40f5059f7056`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:56.224583 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2c8c98a2-7f7d-4012-816d-73695583d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:56.267229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ab2db-0d89-4480-a2d8-47aeee445370 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:56.328128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:56.335695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1036/2058] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:56.344233 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c98a2-7f7d-4012-816d-73695583d2d2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:56.346248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2c8c98a2-7f7d-4012-816d-73695583d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:56.355611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaddcc3-511a-41c7-aac5-6f7d0fdd4b28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:56.356334 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaddcc3-511a-41c7-aac5-6f7d0fdd4b28 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:56.356663 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaddcc3-511a-41c7-aac5-6f7d0fdd4b28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:56.356944 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaddcc3-511a-41c7-aac5-6f7d0fdd4b28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:56.367850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c98a2-7f7d-4012-816d-73695583d2d2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:56.372853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb240a-fb2b-4a55-a635-e9cf2a4e374c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:56.373760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb240a-fb2b-4a55-a635-e9cf2a4e374c service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:56.374350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb240a-fb2b-4a55-a635-e9cf2a4e374c service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:56.374821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb240a-fb2b-4a55-a635-e9cf2a4e374c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:56.387535 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb240a-fb2b-4a55-a635-e9cf2a4e374c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:56.389306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb240a-fb2b-4a55-a635-e9cf2a4e374c service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:56.403712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1037/2059] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:56.404585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb240a-fb2b-4a55-a635-e9cf2a4e374c service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:56.406691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b1bb240a-fb2b-4a55-a635-e9cf2a4e374c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:56.430140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaddcc3-511a-41c7-aac5-6f7d0fdd4b28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:56.431580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaddcc3-511a-41c7-aac5-6f7d0fdd4b28 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:56.442342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1023/2060] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:56.462194 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:56.515849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb240a-fb2b-4a55-a635-e9cf2a4e374c service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:56.521628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:56.523614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:56.524086 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:56.524737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] PATH_INFO: `/v3/users/6bc813ffe96a45f4b49087c4fe12e823` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:56.526289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc813ffe96a45f4b49087c4fe12e823)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:56.529676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:56.605786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc813ffe96a45f4b49087c4fe12e823'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:56.606643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc813ffe96a45f4b49087c4fe12e823'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:56.607287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc813ffe96a45f4b49087c4fe12e823'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:56.646475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc813ffe96a45f4b49087c4fe12e823'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:56.664612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc813ffe96a45f4b49087c4fe12e823'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:56.690572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1038/2061] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:56 2022] DELETE /identity/v3/users/6bc813ffe96a45f4b49087c4fe12e823 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:57.080646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaddcc3-511a-41c7-aac5-6f7d0fdd4b28 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:57.097621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaddcc3-511a-41c7-aac5-6f7d0fdd4b28 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:57.205954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:57.207300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:57.207631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:57.207981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] PATH_INFO: `/v3/projects/8ade814d33ff4327b2d47b4f5f8ab168` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:57.210087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ade814d33ff4327b2d47b4f5f8ab168)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:57.212519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:57.295461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ade814d33ff4327b2d47b4f5f8ab168', 'request_id': 'req-f37e9ccf-281d-479b-91c3-256bdde0e0fe'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.319848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ade814d33ff4327b2d47b4f5f8ab168', 'request_id': 'req-f37e9ccf-281d-479b-91c3-256bdde0e0fe'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.330710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ade814d33ff4327b2d47b4f5f8ab168', 'request_id': 'req-f37e9ccf-281d-479b-91c3-256bdde0e0fe'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.346183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] The token cache is being invalidate because project 8ade814d33ff4327b2d47b4f5f8ab168 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:57.346773 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.347828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.351367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1024/2062] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:57 2022] DELETE /identity/v3/projects/8ade814d33ff4327b2d47b4f5f8ab168 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:57.355796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:57.358241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:57.378925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40727d6d-aba8-4d6a-8a03-f5262edeb800 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:57.455302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b5944-8b9d-4a18-91c8-d46bb892ed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:57.456229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b5944-8b9d-4a18-91c8-d46bb892ed9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:57.456550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b5944-8b9d-4a18-91c8-d46bb892ed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:57.456801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b5944-8b9d-4a18-91c8-d46bb892ed9e admin admin] PATH_INFO: `/v3/users/3abc9cd378424bb1a7d4b38cf21fd0fb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:57.458048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b5944-8b9d-4a18-91c8-d46bb892ed9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3abc9cd378424bb1a7d4b38cf21fd0fb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:57.460586 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b5944-8b9d-4a18-91c8-d46bb892ed9e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:57.516369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af3b5944-8b9d-4a18-91c8-d46bb892ed9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3abc9cd378424bb1a7d4b38cf21fd0fb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.517193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af3b5944-8b9d-4a18-91c8-d46bb892ed9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3abc9cd378424bb1a7d4b38cf21fd0fb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.517880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af3b5944-8b9d-4a18-91c8-d46bb892ed9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3abc9cd378424bb1a7d4b38cf21fd0fb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.519273 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:57.535054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af3b5944-8b9d-4a18-91c8-d46bb892ed9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3abc9cd378424bb1a7d4b38cf21fd0fb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.544232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af3b5944-8b9d-4a18-91c8-d46bb892ed9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3abc9cd378424bb1a7d4b38cf21fd0fb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.564466 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1039/2063] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:57 2022] DELETE /identity/v3/users/3abc9cd378424bb1a7d4b38cf21fd0fb => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:57.761079 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:57.762972 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:57.799287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e9ccf-281d-479b-91c3-256bdde0e0fe admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:57.849150 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:57.894434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5effe-0bb1-4df0-b364-cecad921df63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:57.895305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5effe-0bb1-4df0-b364-cecad921df63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:57.895590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5effe-0bb1-4df0-b364-cecad921df63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:57.895904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5effe-0bb1-4df0-b364-cecad921df63 admin admin] PATH_INFO: `/v3/projects/0182c426a5a14ef7825c364740ead0ed` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:57.897743 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5effe-0bb1-4df0-b364-cecad921df63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0182c426a5a14ef7825c364740ead0ed)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:57.899642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5effe-0bb1-4df0-b364-cecad921df63 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:57.951867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2ec5effe-0bb1-4df0-b364-cecad921df63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0182c426a5a14ef7825c364740ead0ed', 'request_id': 'req-2ec5effe-0bb1-4df0-b364-cecad921df63'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.961265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2ec5effe-0bb1-4df0-b364-cecad921df63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0182c426a5a14ef7825c364740ead0ed', 'request_id': 'req-2ec5effe-0bb1-4df0-b364-cecad921df63'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.979452 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2ec5effe-0bb1-4df0-b364-cecad921df63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0182c426a5a14ef7825c364740ead0ed', 'request_id': 'req-2ec5effe-0bb1-4df0-b364-cecad921df63'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.993936 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2ec5effe-0bb1-4df0-b364-cecad921df63 admin admin] The token cache is being invalidate because project 0182c426a5a14ef7825c364740ead0ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:02:57.994232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2ec5effe-0bb1-4df0-b364-cecad921df63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.995068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2ec5effe-0bb1-4df0-b364-cecad921df63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:02:57.997474 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1025/2064] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:02:57 2022] DELETE /identity/v3/projects/0182c426a5a14ef7825c364740ead0ed => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:02:58.003724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949088dd-9736-4499-87a3-c1cfb485cb48 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.004023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949088dd-9736-4499-87a3-c1cfb485cb48 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.004308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949088dd-9736-4499-87a3-c1cfb485cb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.041860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-949088dd-9736-4499-87a3-c1cfb485cb48 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:58.049643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-949088dd-9736-4499-87a3-c1cfb485cb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:58.104691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0f945-f4f8-4053-8e88-533cf08a19e6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.104979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0f945-f4f8-4053-8e88-533cf08a19e6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.105260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0f945-f4f8-4053-8e88-533cf08a19e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.127366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ccb0f945-f4f8-4053-8e88-533cf08a19e6 None None] MFA Rules not processed for user `1b0423a12e39453e9cd6d1e8ab43af60`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:58.128758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ccb0f945-f4f8-4053-8e88-533cf08a19e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:58.138645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1040/2065] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:58.146391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf303c-e5f8-48b4-afd6-d6f52df1f8ef None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.146760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf303c-e5f8-48b4-afd6-d6f52df1f8ef None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.147642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf303c-e5f8-48b4-afd6-d6f52df1f8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.178738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-addf303c-e5f8-48b4-afd6-d6f52df1f8ef None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:58.180241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-addf303c-e5f8-48b4-afd6-d6f52df1f8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:58.189930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1041/2066] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:58.193855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf303c-e5f8-48b4-afd6-d6f52df1f8ef None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:58.204804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf303c-e5f8-48b4-afd6-d6f52df1f8ef None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:58.210151 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1026/2067] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:58.211744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cbff1-7973-4169-b871-5d9ceb711f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:58.212487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cbff1-7973-4169-b871-5d9ceb711f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.212837 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cbff1-7973-4169-b871-5d9ceb711f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.213199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cbff1-7973-4169-b871-5d9ceb711f09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.214358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49cbff1-7973-4169-b871-5d9ceb711f09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:02:58.214739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49cbff1-7973-4169-b871-5d9ceb711f09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:58.217081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49cbff1-7973-4169-b871-5d9ceb711f09 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:58.238305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1042/2068] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:02:58.239188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0f945-f4f8-4053-8e88-533cf08a19e6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:58.250671 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0f945-f4f8-4053-8e88-533cf08a19e6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:58.257249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9669b-c574-414c-b422-362a5931669b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:58.258724 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9669b-c574-414c-b422-362a5931669b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.259036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9669b-c574-414c-b422-362a5931669b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.259574 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9669b-c574-414c-b422-362a5931669b admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.260342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9669b-c574-414c-b422-362a5931669b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:58.263489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9669b-c574-414c-b422-362a5931669b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:58.295102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1027/2069] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:02:58 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:58.302072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cbff1-7973-4169-b871-5d9ceb711f09 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:58.314520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cbff1-7973-4169-b871-5d9ceb711f09 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:58.404450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8c226-981b-4f3c-b2eb-7a3e7ee95a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:58.405142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8c226-981b-4f3c-b2eb-7a3e7ee95a4f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.405438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8c226-981b-4f3c-b2eb-7a3e7ee95a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.405698 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8c226-981b-4f3c-b2eb-7a3e7ee95a4f admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.406395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8c226-981b-4f3c-b2eb-7a3e7ee95a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:58.408367 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8c226-981b-4f3c-b2eb-7a3e7ee95a4f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:58.437308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1043/2070] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:02:58 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:02:58.441770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9669b-c574-414c-b422-362a5931669b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:58.458740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9669b-c574-414c-b422-362a5931669b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:58.467836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802394b-9bf1-4296-a5fa-679c22c4b3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:58.468610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802394b-9bf1-4296-a5fa-679c22c4b3ec admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.469049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802394b-9bf1-4296-a5fa-679c22c4b3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.469435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802394b-9bf1-4296-a5fa-679c22c4b3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.470277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802394b-9bf1-4296-a5fa-679c22c4b3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:58.472123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802394b-9bf1-4296-a5fa-679c22c4b3ec admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:58.490062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1028/2071] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:58.491433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8c226-981b-4f3c-b2eb-7a3e7ee95a4f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:58.502235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8c226-981b-4f3c-b2eb-7a3e7ee95a4f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:58.511469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b454fcf-1769-4143-b121-64f946cc0ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:58.512277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b454fcf-1769-4143-b121-64f946cc0ffe admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.512661 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b454fcf-1769-4143-b121-64f946cc0ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.513017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b454fcf-1769-4143-b121-64f946cc0ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.513842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b454fcf-1769-4143-b121-64f946cc0ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:58.515622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b454fcf-1769-4143-b121-64f946cc0ffe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:58.536298 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1044/2072] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:02:58.537455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802394b-9bf1-4296-a5fa-679c22c4b3ec admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:58.550334 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802394b-9bf1-4296-a5fa-679c22c4b3ec admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:58.560464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763ee05-73fc-46a0-89e6-073c9a945119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:58.562405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763ee05-73fc-46a0-89e6-073c9a945119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.562880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763ee05-73fc-46a0-89e6-073c9a945119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.563468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763ee05-73fc-46a0-89e6-073c9a945119 admin admin] PATH_INFO: `/v3/projects/1da9afa4daa046239d16a594150a994c/users/17071129652c4fa09b3b4b20c0dc0725/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.588554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763ee05-73fc-46a0-89e6-073c9a945119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da9afa4daa046239d16a594150a994c, user_id=17071129652c4fa09b3b4b20c0dc0725, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:58.593096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763ee05-73fc-46a0-89e6-073c9a945119 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:58.604441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:58.616187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1029/2073] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:02:58 2022] PUT /identity/v3/projects/1da9afa4daa046239d16a594150a994c/users/17071129652c4fa09b3b4b20c0dc0725/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:02:58.623503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943648b0-cba3-4477-946d-76cb60ce41ac None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.623873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943648b0-cba3-4477-946d-76cb60ce41ac None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.624603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943648b0-cba3-4477-946d-76cb60ce41ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.669242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-943648b0-cba3-4477-946d-76cb60ce41ac None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:58.671132 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-943648b0-cba3-4477-946d-76cb60ce41ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:58.707040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:58.795060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1045/2074] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:58.805708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763ee05-73fc-46a0-89e6-073c9a945119 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:58.807478 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d763ee05-73fc-46a0-89e6-073c9a945119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:58.939513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763ee05-73fc-46a0-89e6-073c9a945119 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:58.956229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c6c22-8bc2-464b-8980-b08dc3a063cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:58.957176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c6c22-8bc2-464b-8980-b08dc3a063cb service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.957552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c6c22-8bc2-464b-8980-b08dc3a063cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.957908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c6c22-8bc2-464b-8980-b08dc3a063cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.973102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c6c22-8bc2-464b-8980-b08dc3a063cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:58.974654 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c6c22-8bc2-464b-8980-b08dc3a063cb service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:58.994847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1030/2075] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:59.987671 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943648b0-cba3-4477-946d-76cb60ce41ac None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:59.989198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-943648b0-cba3-4477-946d-76cb60ce41ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:00.001718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943648b0-cba3-4477-946d-76cb60ce41ac None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:00.007951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:00.009310 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:00.009779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:00.010247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] PATH_INFO: `/v3/users/9337c184fac44c6c815f40f5059f7056` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:00.011912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9337c184fac44c6c815f40f5059f7056)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:00.014085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:00.089690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9337c184fac44c6c815f40f5059f7056'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:00.090552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9337c184fac44c6c815f40f5059f7056'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:00.091870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9337c184fac44c6c815f40f5059f7056'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:00.115476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9337c184fac44c6c815f40f5059f7056'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:00.135079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9337c184fac44c6c815f40f5059f7056'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:00.155085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1046/2076] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:02:59 2022] DELETE /identity/v3/users/9337c184fac44c6c815f40f5059f7056 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:00.159960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb2a1c-2bfd-48a4-b70a-82dd2b040d3b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:00.160507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb2a1c-2bfd-48a4-b70a-82dd2b040d3b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:00.160609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb2a1c-2bfd-48a4-b70a-82dd2b040d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:00.180320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-86bb2a1c-2bfd-48a4-b70a-82dd2b040d3b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:00.181570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-86bb2a1c-2bfd-48a4-b70a-82dd2b040d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:00.248924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1031/2077] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:00.259393 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:00.260902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:00.363808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ea0d9-a66f-4b90-8e65-0f91b73520c5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:00.368641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a7db6-0886-4adb-bf29-92b67c5b8ca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:00.369356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a7db6-0886-4adb-bf29-92b67c5b8ca0 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:00.369643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a7db6-0886-4adb-bf29-92b67c5b8ca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:00.369921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a7db6-0886-4adb-bf29-92b67c5b8ca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:00.387822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a7db6-0886-4adb-bf29-92b67c5b8ca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:00.389961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a7db6-0886-4adb-bf29-92b67c5b8ca0 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:00.403220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1047/2078] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:01.301243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb2a1c-2bfd-48a4-b70a-82dd2b040d3b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:01.303368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-86bb2a1c-2bfd-48a4-b70a-82dd2b040d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:01.348723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb2a1c-2bfd-48a4-b70a-82dd2b040d3b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:01.368861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000c178-e5e8-4780-a839-20035a40435c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:01.370272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000c178-e5e8-4780-a839-20035a40435c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:01.370757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000c178-e5e8-4780-a839-20035a40435c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:01.371339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000c178-e5e8-4780-a839-20035a40435c admin admin] PATH_INFO: `/v3/projects/218e8654a93f4ff1b3125c91f8fc892c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:01.375317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000c178-e5e8-4780-a839-20035a40435c admin admin] RBAC: Authorizing `identity:delete_project(project_id=218e8654a93f4ff1b3125c91f8fc892c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:01.379852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000c178-e5e8-4780-a839-20035a40435c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:01.498685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4000c178-e5e8-4780-a839-20035a40435c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '218e8654a93f4ff1b3125c91f8fc892c', 'request_id': 'req-4000c178-e5e8-4780-a839-20035a40435c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:01.513590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4000c178-e5e8-4780-a839-20035a40435c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '218e8654a93f4ff1b3125c91f8fc892c', 'request_id': 'req-4000c178-e5e8-4780-a839-20035a40435c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:01.531119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4000c178-e5e8-4780-a839-20035a40435c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '218e8654a93f4ff1b3125c91f8fc892c', 'request_id': 'req-4000c178-e5e8-4780-a839-20035a40435c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:01.579990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4000c178-e5e8-4780-a839-20035a40435c admin admin] The token cache is being invalidate because project 218e8654a93f4ff1b3125c91f8fc892c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:03:01.579990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4000c178-e5e8-4780-a839-20035a40435c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:01.581810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06a841-3f0f-43a6-8893-e11bdf69c0ed None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:01.582271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06a841-3f0f-43a6-8893-e11bdf69c0ed None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:01.582561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06a841-3f0f-43a6-8893-e11bdf69c0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:01.588626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4000c178-e5e8-4780-a839-20035a40435c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:01.596324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1032/2079] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:03:01 2022] DELETE /identity/v3/projects/218e8654a93f4ff1b3125c91f8fc892c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:01.597807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724f387-3e6a-4917-ad47-4f0279ca026c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:01.598100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724f387-3e6a-4917-ad47-4f0279ca026c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:01.598378 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724f387-3e6a-4917-ad47-4f0279ca026c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:01.622910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0724f387-3e6a-4917-ad47-4f0279ca026c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:01.625018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0724f387-3e6a-4917-ad47-4f0279ca026c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:01.655629 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:01.655921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-be06a841-3f0f-43a6-8893-e11bdf69c0ed None None] MFA Rules not processed for user `17071129652c4fa09b3b4b20c0dc0725`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:01.657649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-be06a841-3f0f-43a6-8893-e11bdf69c0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:01.699600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:01.735163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1033/2080] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:01.743313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49156449-bcac-4f2d-823e-36bc1d899f48 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:01.743871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49156449-bcac-4f2d-823e-36bc1d899f48 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:01.745458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49156449-bcac-4f2d-823e-36bc1d899f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:01.748307 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1048/2081] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:01.760917 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06a841-3f0f-43a6-8893-e11bdf69c0ed None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:01.762695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-be06a841-3f0f-43a6-8893-e11bdf69c0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:01.798427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-49156449-bcac-4f2d-823e-36bc1d899f48 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:01.799701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-49156449-bcac-4f2d-823e-36bc1d899f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:01.827013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1034/2082] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:03:01.831515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49156449-bcac-4f2d-823e-36bc1d899f48 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:01.848931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49156449-bcac-4f2d-823e-36bc1d899f48 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:01.863117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28918930-4da5-4bcf-aee5-7ce60b4efd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:01.863580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28918930-4da5-4bcf-aee5-7ce60b4efd1b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:01.863892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28918930-4da5-4bcf-aee5-7ce60b4efd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:01.864552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28918930-4da5-4bcf-aee5-7ce60b4efd1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:01.865300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28918930-4da5-4bcf-aee5-7ce60b4efd1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:03:01.868725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28918930-4da5-4bcf-aee5-7ce60b4efd1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:01.871749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28918930-4da5-4bcf-aee5-7ce60b4efd1b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:01.889107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06a841-3f0f-43a6-8893-e11bdf69c0ed None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:01.893075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f119f9-2e92-413d-93af-10ef1aa93cd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:01.893760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f119f9-2e92-413d-93af-10ef1aa93cd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:01.894047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f119f9-2e92-413d-93af-10ef1aa93cd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:01.894906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f119f9-2e92-413d-93af-10ef1aa93cd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:01.907273 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f119f9-2e92-413d-93af-10ef1aa93cd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:01.916785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f119f9-2e92-413d-93af-10ef1aa93cd7 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:01.921518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1035/2083] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:03:01.926971 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28918930-4da5-4bcf-aee5-7ce60b4efd1b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:01.944865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1049/2084] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:03:01.964858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28918930-4da5-4bcf-aee5-7ce60b4efd1b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:01.973235 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f188ad-909e-4acd-81ff-8d6ee58ccd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:01.974034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f188ad-909e-4acd-81ff-8d6ee58ccd6d admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:01.974389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f188ad-909e-4acd-81ff-8d6ee58ccd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:01.974776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f188ad-909e-4acd-81ff-8d6ee58ccd6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:01.975681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f188ad-909e-4acd-81ff-8d6ee58ccd6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:01.977718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f188ad-909e-4acd-81ff-8d6ee58ccd6d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:02.020281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1036/2085] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:03:01 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:02.028122 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f119f9-2e92-413d-93af-10ef1aa93cd7 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:02.048031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f119f9-2e92-413d-93af-10ef1aa93cd7 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:02.122735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edda48b-4635-4982-91dd-5563b641cfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:02.123419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edda48b-4635-4982-91dd-5563b641cfa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:02.123872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edda48b-4635-4982-91dd-5563b641cfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:02.124156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edda48b-4635-4982-91dd-5563b641cfa0 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:02.124857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edda48b-4635-4982-91dd-5563b641cfa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:02.127085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edda48b-4635-4982-91dd-5563b641cfa0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:02.152463 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1050/2086] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:03:02 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:02.156765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f188ad-909e-4acd-81ff-8d6ee58ccd6d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:02.170135 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f188ad-909e-4acd-81ff-8d6ee58ccd6d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:02.175964 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d96f4c-a2aa-4dce-9055-909fc2ae6a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:02.176810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d96f4c-a2aa-4dce-9055-909fc2ae6a92 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:02.177096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d96f4c-a2aa-4dce-9055-909fc2ae6a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:02.177373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d96f4c-a2aa-4dce-9055-909fc2ae6a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:02.178141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d96f4c-a2aa-4dce-9055-909fc2ae6a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:02.180343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d96f4c-a2aa-4dce-9055-909fc2ae6a92 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:02.195465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1037/2087] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:03:02.199416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edda48b-4635-4982-91dd-5563b641cfa0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:02.211813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edda48b-4635-4982-91dd-5563b641cfa0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:02.222442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd780416-d45c-45f7-9785-c4f42dbd71cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:02.223869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd780416-d45c-45f7-9785-c4f42dbd71cd admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:02.224157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd780416-d45c-45f7-9785-c4f42dbd71cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:02.224485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd780416-d45c-45f7-9785-c4f42dbd71cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:02.225768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd780416-d45c-45f7-9785-c4f42dbd71cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:02.227429 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd780416-d45c-45f7-9785-c4f42dbd71cd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:02.245187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1051/2088] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:03:02.248112 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d96f4c-a2aa-4dce-9055-909fc2ae6a92 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:02.260810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d96f4c-a2aa-4dce-9055-909fc2ae6a92 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:02.266203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45545978-f04c-4285-bc25-d236c5b4b55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:02.267422 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45545978-f04c-4285-bc25-d236c5b4b55f admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:02.267843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45545978-f04c-4285-bc25-d236c5b4b55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:02.268221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45545978-f04c-4285-bc25-d236c5b4b55f admin admin] PATH_INFO: `/v3/projects/ab4832b13b5d4a908057db33289b73e9/users/255102c6c13a4911b69d504c77971f43/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:02.302994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45545978-f04c-4285-bc25-d236c5b4b55f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab4832b13b5d4a908057db33289b73e9, user_id=255102c6c13a4911b69d504c77971f43, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:02.307068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45545978-f04c-4285-bc25-d236c5b4b55f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:02.342252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1038/2089] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:03:02 2022] PUT /identity/v3/projects/ab4832b13b5d4a908057db33289b73e9/users/255102c6c13a4911b69d504c77971f43/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:02.354100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55f69-b64b-40f1-87a6-5ad64efff620 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:02.354494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55f69-b64b-40f1-87a6-5ad64efff620 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:02.354774 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55f69-b64b-40f1-87a6-5ad64efff620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:02.388265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fdc55f69-b64b-40f1-87a6-5ad64efff620 None None] MFA Rules not processed for user `255102c6c13a4911b69d504c77971f43`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:02.390014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fdc55f69-b64b-40f1-87a6-5ad64efff620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:02.449028 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:02.522552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1052/2090] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:02.526916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45545978-f04c-4285-bc25-d236c5b4b55f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:02.610863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45545978-f04c-4285-bc25-d236c5b4b55f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:02.616173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f0002-a1ba-4711-9fdb-8e06ceaa81b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:02.617133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f0002-a1ba-4711-9fdb-8e06ceaa81b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:02.618031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f0002-a1ba-4711-9fdb-8e06ceaa81b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:02.618512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f0002-a1ba-4711-9fdb-8e06ceaa81b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:02.619495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f0002-a1ba-4711-9fdb-8e06ceaa81b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:02.622290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f0002-a1ba-4711-9fdb-8e06ceaa81b8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:02.679699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1039/2091] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:03:02 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:03:02.681860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc55f69-b64b-40f1-87a6-5ad64efff620 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:02.703614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc55f69-b64b-40f1-87a6-5ad64efff620 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:02.791945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7003f3-c103-4b4b-bb91-3cc712e7029a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:02.792608 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7003f3-c103-4b4b-bb91-3cc712e7029a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:02.793334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7003f3-c103-4b4b-bb91-3cc712e7029a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:02.793334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7003f3-c103-4b4b-bb91-3cc712e7029a admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:02.794011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7003f3-c103-4b4b-bb91-3cc712e7029a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:02.796229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7003f3-c103-4b4b-bb91-3cc712e7029a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:02.821849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1053/2092] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:03:02 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:02.823052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f0002-a1ba-4711-9fdb-8e06ceaa81b8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:02.833829 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f0002-a1ba-4711-9fdb-8e06ceaa81b8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:02.839703 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b915d-357c-4b6c-86a9-b3cece4674ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:02.840224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b915d-357c-4b6c-86a9-b3cece4674ee admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:02.840512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b915d-357c-4b6c-86a9-b3cece4674ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:02.840792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b915d-357c-4b6c-86a9-b3cece4674ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:02.841534 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b915d-357c-4b6c-86a9-b3cece4674ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:02.843138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b915d-357c-4b6c-86a9-b3cece4674ee admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:02.855839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1040/2093] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:03:02.859688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7003f3-c103-4b4b-bb91-3cc712e7029a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:02.870295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7003f3-c103-4b4b-bb91-3cc712e7029a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:02.879442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dce54b-2492-45de-b9a7-65e4cb0a68ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:02.879710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dce54b-2492-45de-b9a7-65e4cb0a68ef admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:02.879833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dce54b-2492-45de-b9a7-65e4cb0a68ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:02.880107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dce54b-2492-45de-b9a7-65e4cb0a68ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:02.880849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dce54b-2492-45de-b9a7-65e4cb0a68ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:02.883121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dce54b-2492-45de-b9a7-65e4cb0a68ef admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:02.903106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1054/2094] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:03:02.907464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b915d-357c-4b6c-86a9-b3cece4674ee admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:02.919109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b915d-357c-4b6c-86a9-b3cece4674ee admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:02.925287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfcd8b-13a2-40dd-ae4b-5c51f149d493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:02.926400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfcd8b-13a2-40dd-ae4b-5c51f149d493 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:02.926978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfcd8b-13a2-40dd-ae4b-5c51f149d493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:02.927229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfcd8b-13a2-40dd-ae4b-5c51f149d493 admin admin] PATH_INFO: `/v3/projects/4db67d7ff5084437aceaef2fead7e2c3/users/4cf05c3a62b24ca9ac7bb5a1424b60d0/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:02.957368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfcd8b-13a2-40dd-ae4b-5c51f149d493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4db67d7ff5084437aceaef2fead7e2c3, user_id=4cf05c3a62b24ca9ac7bb5a1424b60d0, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:02.962517 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfcd8b-13a2-40dd-ae4b-5c51f149d493 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:02.973801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:02.982049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1041/2095] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:03:02 2022] PUT /identity/v3/projects/4db67d7ff5084437aceaef2fead7e2c3/users/4cf05c3a62b24ca9ac7bb5a1424b60d0/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:02.992176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fb073-a884-4140-b6ea-50e8b472b38d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:02.992573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fb073-a884-4140-b6ea-50e8b472b38d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:02.992957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fb073-a884-4140-b6ea-50e8b472b38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.030273 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-788fb073-a884-4140-b6ea-50e8b472b38d None None] MFA Rules not processed for user `4cf05c3a62b24ca9ac7bb5a1424b60d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:03.031867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-788fb073-a884-4140-b6ea-50e8b472b38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:03.130181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1055/2096] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:03.134685 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfcd8b-13a2-40dd-ae4b-5c51f149d493 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:03.204852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfcd8b-13a2-40dd-ae4b-5c51f149d493 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:03.215463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042113c3-fdd4-4352-8bef-658b7c2a109b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:03.216091 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042113c3-fdd4-4352-8bef-658b7c2a109b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:03.220034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042113c3-fdd4-4352-8bef-658b7c2a109b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:03.220516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042113c3-fdd4-4352-8bef-658b7c2a109b admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.222153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042113c3-fdd4-4352-8bef-658b7c2a109b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:03.223891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042113c3-fdd4-4352-8bef-658b7c2a109b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:03.260267 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1042/2097] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:03:03 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:03.264513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fb073-a884-4140-b6ea-50e8b472b38d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:03.277399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fb073-a884-4140-b6ea-50e8b472b38d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:03.353992 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:03.367262 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd628e58-5b97-431d-8b02-3e42bd232e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:03.368033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd628e58-5b97-431d-8b02-3e42bd232e09 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:03.368385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd628e58-5b97-431d-8b02-3e42bd232e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:03.368666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd628e58-5b97-431d-8b02-3e42bd232e09 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.369392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd628e58-5b97-431d-8b02-3e42bd232e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:03.371324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd628e58-5b97-431d-8b02-3e42bd232e09 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:03.394828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1056/2098] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:03:03 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:03.400095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042113c3-fdd4-4352-8bef-658b7c2a109b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:03.417907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042113c3-fdd4-4352-8bef-658b7c2a109b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:03.423559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b1c3d-1db6-4d4a-88ef-b4be2879fcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:03.424265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b1c3d-1db6-4d4a-88ef-b4be2879fcd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:03.424551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b1c3d-1db6-4d4a-88ef-b4be2879fcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:03.424831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b1c3d-1db6-4d4a-88ef-b4be2879fcd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.425569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b1c3d-1db6-4d4a-88ef-b4be2879fcd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:03.427550 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b1c3d-1db6-4d4a-88ef-b4be2879fcd7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:03.434486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc5a8c-bcee-4d96-9b9f-9c8cba23766e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:03.434797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc5a8c-bcee-4d96-9b9f-9c8cba23766e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:03.435059 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc5a8c-bcee-4d96-9b9f-9c8cba23766e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.447233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1043/2099] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:03:03.448628 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b1c3d-1db6-4d4a-88ef-b4be2879fcd7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:03.457363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2dcc5a8c-bcee-4d96-9b9f-9c8cba23766e None None] MFA Rules not processed for user `17071129652c4fa09b3b4b20c0dc0725`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:03.458684 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2dcc5a8c-bcee-4d96-9b9f-9c8cba23766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:03.460163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b1c3d-1db6-4d4a-88ef-b4be2879fcd7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:03.469161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944597e-a166-4d21-96f6-9038762e69b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:03.470013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944597e-a166-4d21-96f6-9038762e69b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:03.470295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944597e-a166-4d21-96f6-9038762e69b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:03.470573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944597e-a166-4d21-96f6-9038762e69b6 admin admin] PATH_INFO: `/v3/projects/9b9a5aa1262f43aa83c3c9f56e60cea4/users/b409a3e229054f0b987527c0665ee276/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.486866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944597e-a166-4d21-96f6-9038762e69b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9a5aa1262f43aa83c3c9f56e60cea4, user_id=b409a3e229054f0b987527c0665ee276, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:03.489578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944597e-a166-4d21-96f6-9038762e69b6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:03.505390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1044/2100] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:03:03 2022] PUT /identity/v3/projects/9b9a5aa1262f43aa83c3c9f56e60cea4/users/b409a3e229054f0b987527c0665ee276/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:03:03.509246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944597e-a166-4d21-96f6-9038762e69b6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:03.520924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1057/2101] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:03:03.586865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944597e-a166-4d21-96f6-9038762e69b6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:03.592936 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628b887-fe8d-4fa5-9729-23be0c951fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:03.593678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628b887-fe8d-4fa5-9729-23be0c951fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:03.595573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628b887-fe8d-4fa5-9729-23be0c951fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:03.595573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628b887-fe8d-4fa5-9729-23be0c951fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.595573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628b887-fe8d-4fa5-9729-23be0c951fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:03.596866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628b887-fe8d-4fa5-9729-23be0c951fbf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:03.610143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1045/2102] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:03:03.611552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc5a8c-bcee-4d96-9b9f-9c8cba23766e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:03.623384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc5a8c-bcee-4d96-9b9f-9c8cba23766e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:03.628014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02f29d-56f1-4ef0-90d3-a2846c9a84e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:03.628910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02f29d-56f1-4ef0-90d3-a2846c9a84e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:03.629323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02f29d-56f1-4ef0-90d3-a2846c9a84e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:03.629608 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02f29d-56f1-4ef0-90d3-a2846c9a84e0 admin admin] PATH_INFO: `/v3/projects/9b9a5aa1262f43aa83c3c9f56e60cea4/users/b409a3e229054f0b987527c0665ee276/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.632195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02f29d-56f1-4ef0-90d3-a2846c9a84e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9a5aa1262f43aa83c3c9f56e60cea4, user_id=b409a3e229054f0b987527c0665ee276, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:03.634894 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02f29d-56f1-4ef0-90d3-a2846c9a84e0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:03.645498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1058/2103] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:03:03 2022] PUT /identity/v3/projects/9b9a5aa1262f43aa83c3c9f56e60cea4/users/b409a3e229054f0b987527c0665ee276/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:03:03.657393 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375c551-b667-421e-834d-bf0ac40e3d82 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:03.657812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375c551-b667-421e-834d-bf0ac40e3d82 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:03.658227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375c551-b667-421e-834d-bf0ac40e3d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.686146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c375c551-b667-421e-834d-bf0ac40e3d82 None None] MFA Rules not processed for user `b409a3e229054f0b987527c0665ee276`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:03.687397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c375c551-b667-421e-834d-bf0ac40e3d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:03.723128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:03.766636 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1046/2104] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:03.777956 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02f29d-56f1-4ef0-90d3-a2846c9a84e0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:03.842887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02f29d-56f1-4ef0-90d3-a2846c9a84e0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:03.846562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673405e-a589-4af8-a66f-8ee598b0af4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:03.847208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673405e-a589-4af8-a66f-8ee598b0af4c service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:03.847525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673405e-a589-4af8-a66f-8ee598b0af4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:03.847841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673405e-a589-4af8-a66f-8ee598b0af4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.909021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1673405e-a589-4af8-a66f-8ee598b0af4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:03.910563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1673405e-a589-4af8-a66f-8ee598b0af4c service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:03.921194 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1059/2105] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:05.534829 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375c551-b667-421e-834d-bf0ac40e3d82 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:05.536288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c375c551-b667-421e-834d-bf0ac40e3d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:05.548129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375c551-b667-421e-834d-bf0ac40e3d82 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:05.551711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c98e0-9b3b-4d73-9636-15c51abd2a6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:05.552358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c98e0-9b3b-4d73-9636-15c51abd2a6c service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:05.552656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c98e0-9b3b-4d73-9636-15c51abd2a6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:05.553086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c98e0-9b3b-4d73-9636-15c51abd2a6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:05.614779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c98e0-9b3b-4d73-9636-15c51abd2a6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:05.617492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c98e0-9b3b-4d73-9636-15c51abd2a6c service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:05.628505 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1047/2106] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:06.254794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673405e-a589-4af8-a66f-8ee598b0af4c service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:06.263319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673405e-a589-4af8-a66f-8ee598b0af4c service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:06.266939 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb360b-d29e-4297-8724-f6b2be1dc67a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:06.267575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb360b-d29e-4297-8724-f6b2be1dc67a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:06.267893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb360b-d29e-4297-8724-f6b2be1dc67a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:06.268173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb360b-d29e-4297-8724-f6b2be1dc67a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:06.277764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb360b-d29e-4297-8724-f6b2be1dc67a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:06.279384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb360b-d29e-4297-8724-f6b2be1dc67a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:06.290380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1060/2107] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:10.238264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c98e0-9b3b-4d73-9636-15c51abd2a6c service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:10.334250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c98e0-9b3b-4d73-9636-15c51abd2a6c service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:10.341302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:10.343005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:10.343363 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:10.343748 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] PATH_INFO: `/v3/users/255102c6c13a4911b69d504c77971f43` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:10.345653 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=255102c6c13a4911b69d504c77971f43)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:10.347948 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:10.411558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '255102c6c13a4911b69d504c77971f43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:10.412369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '255102c6c13a4911b69d504c77971f43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:10.421892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '255102c6c13a4911b69d504c77971f43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:10.435999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:10.438006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '255102c6c13a4911b69d504c77971f43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:10.450312 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '255102c6c13a4911b69d504c77971f43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:10.461501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1048/2108] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:03:10 2022] DELETE /identity/v3/users/255102c6c13a4911b69d504c77971f43 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:10.467148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7e3bb-94fa-4e44-b7e7-801f34a15947 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:10.467957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7e3bb-94fa-4e44-b7e7-801f34a15947 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:10.468479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7e3bb-94fa-4e44-b7e7-801f34a15947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:10.497146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-aac7e3bb-94fa-4e44-b7e7-801f34a15947 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:10.499307 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aac7e3bb-94fa-4e44-b7e7-801f34a15947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:10.506269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:10.588424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1061/2109] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:10.598365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:10.599995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:10.664163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7784c19-e647-428a-be6f-ea2cb8b30078 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:10.667908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733437b6-c725-4b3d-8c3c-729edf46eba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:10.668614 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733437b6-c725-4b3d-8c3c-729edf46eba8 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:10.668961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733437b6-c725-4b3d-8c3c-729edf46eba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:10.669302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733437b6-c725-4b3d-8c3c-729edf46eba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:10.678718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733437b6-c725-4b3d-8c3c-729edf46eba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:10.680274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733437b6-c725-4b3d-8c3c-729edf46eba8 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:10.690553 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1049/2110] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:10.961441 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7e3bb-94fa-4e44-b7e7-801f34a15947 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:10.963030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aac7e3bb-94fa-4e44-b7e7-801f34a15947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:10.996950 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7e3bb-94fa-4e44-b7e7-801f34a15947 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:11.009117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:11.010080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:11.010472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:11.010813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] PATH_INFO: `/v3/projects/ab4832b13b5d4a908057db33289b73e9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:11.012380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab4832b13b5d4a908057db33289b73e9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:11.015095 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:11.074515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab4832b13b5d4a908057db33289b73e9', 'request_id': 'req-24ae194d-028c-4719-9cc3-fda26529f7c5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.076665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab4832b13b5d4a908057db33289b73e9', 'request_id': 'req-24ae194d-028c-4719-9cc3-fda26529f7c5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.082394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab4832b13b5d4a908057db33289b73e9', 'request_id': 'req-24ae194d-028c-4719-9cc3-fda26529f7c5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.096455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] The token cache is being invalidate because project ab4832b13b5d4a908057db33289b73e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:03:11.096724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.097380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.099243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1062/2111] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:03:10 2022] DELETE /identity/v3/projects/ab4832b13b5d4a908057db33289b73e9 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:11.102905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733437b6-c725-4b3d-8c3c-729edf46eba8 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:11.114204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733437b6-c725-4b3d-8c3c-729edf46eba8 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:11.173426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:11.174326 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:11.174608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:11.174911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] PATH_INFO: `/v3/users/4cf05c3a62b24ca9ac7bb5a1424b60d0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:11.176128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf05c3a62b24ca9ac7bb5a1424b60d0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:11.178048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:11.223820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf05c3a62b24ca9ac7bb5a1424b60d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.224449 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf05c3a62b24ca9ac7bb5a1424b60d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.232271 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf05c3a62b24ca9ac7bb5a1424b60d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.244765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf05c3a62b24ca9ac7bb5a1424b60d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.251768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf05c3a62b24ca9ac7bb5a1424b60d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.257634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1050/2112] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:03:11 2022] DELETE /identity/v3/users/4cf05c3a62b24ca9ac7bb5a1424b60d0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:11.520243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:11.521831 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:11.534928 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae194d-028c-4719-9cc3-fda26529f7c5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:11.594841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:11.595739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:11.596020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:11.596296 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] PATH_INFO: `/v3/projects/4db67d7ff5084437aceaef2fead7e2c3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:11.597790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4db67d7ff5084437aceaef2fead7e2c3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:11.599775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:11.644148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:11.658676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4db67d7ff5084437aceaef2fead7e2c3', 'request_id': 'req-1016f7e7-d290-4c3c-9f40-bf812e1f178d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.672769 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4db67d7ff5084437aceaef2fead7e2c3', 'request_id': 'req-1016f7e7-d290-4c3c-9f40-bf812e1f178d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.679418 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4db67d7ff5084437aceaef2fead7e2c3', 'request_id': 'req-1016f7e7-d290-4c3c-9f40-bf812e1f178d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.695113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] The token cache is being invalidate because project 4db67d7ff5084437aceaef2fead7e2c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:03:11.695409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.696073 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.699687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1063/2113] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:03:11 2022] DELETE /identity/v3/projects/4db67d7ff5084437aceaef2fead7e2c3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:03:11.702460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:11.704682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:11.720111 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0638-2591-4e70-a086-460ad494f3d1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:11.724969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:11.799946 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c45e4-5338-4782-ac86-a55c6939e78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:11.800835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c45e4-5338-4782-ac86-a55c6939e78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:11.801121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c45e4-5338-4782-ac86-a55c6939e78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:11.801406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c45e4-5338-4782-ac86-a55c6939e78c admin admin] PATH_INFO: `/v3/users/b409a3e229054f0b987527c0665ee276` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:11.802607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c45e4-5338-4782-ac86-a55c6939e78c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b409a3e229054f0b987527c0665ee276)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:11.804619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c45e4-5338-4782-ac86-a55c6939e78c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:11.849282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a8c45e4-5338-4782-ac86-a55c6939e78c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b409a3e229054f0b987527c0665ee276'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.849921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a8c45e4-5338-4782-ac86-a55c6939e78c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b409a3e229054f0b987527c0665ee276'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.857608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a8c45e4-5338-4782-ac86-a55c6939e78c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b409a3e229054f0b987527c0665ee276'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.869508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a8c45e4-5338-4782-ac86-a55c6939e78c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b409a3e229054f0b987527c0665ee276'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.876480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a8c45e4-5338-4782-ac86-a55c6939e78c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b409a3e229054f0b987527c0665ee276'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:11.882442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1051/2114] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:03:11 2022] DELETE /identity/v3/users/b409a3e229054f0b987527c0665ee276 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:03:12.186288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:12.188467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:12.203387 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016f7e7-d290-4c3c-9f40-bf812e1f178d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:12.290008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:12.290872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:12.291157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:12.291434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c admin admin] PATH_INFO: `/v3/projects/9b9a5aa1262f43aa83c3c9f56e60cea4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:12.292652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b9a5aa1262f43aa83c3c9f56e60cea4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:12.294575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:12.343522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b9a5aa1262f43aa83c3c9f56e60cea4', 'request_id': 'req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:12.349534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b9a5aa1262f43aa83c3c9f56e60cea4', 'request_id': 'req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:12.355253 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b9a5aa1262f43aa83c3c9f56e60cea4', 'request_id': 'req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:12.367732 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c admin admin] The token cache is being invalidate because project 9b9a5aa1262f43aa83c3c9f56e60cea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:03:12.368024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:12.368649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbc30d9c-95c8-4590-b65e-afa3b3e2a37c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:12.370220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1064/2115] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:03:12 2022] DELETE /identity/v3/projects/9b9a5aa1262f43aa83c3c9f56e60cea4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:12.376280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86c538-88ee-4d95-bc00-3c79ce399347 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:12.376680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86c538-88ee-4d95-bc00-3c79ce399347 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:12.376843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86c538-88ee-4d95-bc00-3c79ce399347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:12.395203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-de86c538-88ee-4d95-bc00-3c79ce399347 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:12.396447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-de86c538-88ee-4d95-bc00-3c79ce399347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:12.454599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1052/2116] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:12.462163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cc2ac-a1c8-41d7-a433-17027ba92a43 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:12.462454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cc2ac-a1c8-41d7-a433-17027ba92a43 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:12.462735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cc2ac-a1c8-41d7-a433-17027ba92a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:12.480562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7b8cc2ac-a1c8-41d7-a433-17027ba92a43 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:12.481767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7b8cc2ac-a1c8-41d7-a433-17027ba92a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:12.489643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1065/2117] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:12.494217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86c538-88ee-4d95-bc00-3c79ce399347 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:12.504063 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86c538-88ee-4d95-bc00-3c79ce399347 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:12.510130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf5986-a545-4bca-835e-93f4f3507e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:12.510835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf5986-a545-4bca-835e-93f4f3507e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:12.511117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf5986-a545-4bca-835e-93f4f3507e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:12.511392 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf5986-a545-4bca-835e-93f4f3507e78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:12.512181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf5986-a545-4bca-835e-93f4f3507e78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:03:12.512518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf5986-a545-4bca-835e-93f4f3507e78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:12.514114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf5986-a545-4bca-835e-93f4f3507e78 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:12.525592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1053/2118] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:03:12.529669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cc2ac-a1c8-41d7-a433-17027ba92a43 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:12.538712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cc2ac-a1c8-41d7-a433-17027ba92a43 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:12.543323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85dfea-064c-4a88-80dd-655735b7bafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:12.544001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85dfea-064c-4a88-80dd-655735b7bafe admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:12.544282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85dfea-064c-4a88-80dd-655735b7bafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:12.544556 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85dfea-064c-4a88-80dd-655735b7bafe admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:12.545238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85dfea-064c-4a88-80dd-655735b7bafe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:12.547112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85dfea-064c-4a88-80dd-655735b7bafe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:12.573018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1066/2119] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:03:12 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:12.577141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf5986-a545-4bca-835e-93f4f3507e78 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:12.586928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf5986-a545-4bca-835e-93f4f3507e78 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:12.598458 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:12.644385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893276ce-8c15-48e5-920b-50d7159bdf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:12.645038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893276ce-8c15-48e5-920b-50d7159bdf83 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:12.645318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893276ce-8c15-48e5-920b-50d7159bdf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:12.645592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893276ce-8c15-48e5-920b-50d7159bdf83 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:12.646275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893276ce-8c15-48e5-920b-50d7159bdf83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:12.648222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893276ce-8c15-48e5-920b-50d7159bdf83 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:12.666566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1054/2120] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:03:12 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:12.670592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85dfea-064c-4a88-80dd-655735b7bafe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:12.681384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85dfea-064c-4a88-80dd-655735b7bafe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:12.686001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e3cde-de7c-411e-b1a3-8f1d0da4721f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:12.686648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e3cde-de7c-411e-b1a3-8f1d0da4721f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:12.686934 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e3cde-de7c-411e-b1a3-8f1d0da4721f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:12.687209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e3cde-de7c-411e-b1a3-8f1d0da4721f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:12.687974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e3cde-de7c-411e-b1a3-8f1d0da4721f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:12.689554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e3cde-de7c-411e-b1a3-8f1d0da4721f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:12.699803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1067/2121] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:03:12.703483 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893276ce-8c15-48e5-920b-50d7159bdf83 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:12.713171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893276ce-8c15-48e5-920b-50d7159bdf83 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:12.717664 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647444e-8bde-4f80-b3a9-833de1e6fe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:12.718308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647444e-8bde-4f80-b3a9-833de1e6fe51 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:12.718616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647444e-8bde-4f80-b3a9-833de1e6fe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:12.718893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647444e-8bde-4f80-b3a9-833de1e6fe51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:12.719613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647444e-8bde-4f80-b3a9-833de1e6fe51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:12.721204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647444e-8bde-4f80-b3a9-833de1e6fe51 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:12.731298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1055/2122] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:03:12.735024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e3cde-de7c-411e-b1a3-8f1d0da4721f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:12.745627 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e3cde-de7c-411e-b1a3-8f1d0da4721f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:12.750797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94483ebe-322f-4408-a085-60df1e31d65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:12.751749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94483ebe-322f-4408-a085-60df1e31d65c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:12.752029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94483ebe-322f-4408-a085-60df1e31d65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:12.752301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94483ebe-322f-4408-a085-60df1e31d65c admin admin] PATH_INFO: `/v3/projects/361c479558b7471ca4b92a7727dee1cf/users/cee061b2f23f4e87b472680d87375e80/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:12.764452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:12.767752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94483ebe-322f-4408-a085-60df1e31d65c admin admin] RBAC: Authorizing `identity:create_grant(project_id=361c479558b7471ca4b92a7727dee1cf, user_id=cee061b2f23f4e87b472680d87375e80, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:12.770601 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94483ebe-322f-4408-a085-60df1e31d65c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:12.784431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1068/2123] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:03:12 2022] PUT /identity/v3/projects/361c479558b7471ca4b92a7727dee1cf/users/cee061b2f23f4e87b472680d87375e80/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:12.795292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792ddaa-a834-4679-8d15-e88359fdc96c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:12.795580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792ddaa-a834-4679-8d15-e88359fdc96c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:12.796390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792ddaa-a834-4679-8d15-e88359fdc96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:12.854324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7792ddaa-a834-4679-8d15-e88359fdc96c None None] MFA Rules not processed for user `cee061b2f23f4e87b472680d87375e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:12.855789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7792ddaa-a834-4679-8d15-e88359fdc96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:12.945412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1056/2124] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:12.956334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94483ebe-322f-4408-a085-60df1e31d65c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:12.958029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-94483ebe-322f-4408-a085-60df1e31d65c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:13.013924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94483ebe-322f-4408-a085-60df1e31d65c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:13.017558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6a897-c7e3-44cd-b67b-027f3fe9c1a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:13.018200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6a897-c7e3-44cd-b67b-027f3fe9c1a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:13.018485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6a897-c7e3-44cd-b67b-027f3fe9c1a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:13.018764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6a897-c7e3-44cd-b67b-027f3fe9c1a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:13.027212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6a897-c7e3-44cd-b67b-027f3fe9c1a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:13.028675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6a897-c7e3-44cd-b67b-027f3fe9c1a3 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:13.037319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1069/2125] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:13.062211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792ddaa-a834-4679-8d15-e88359fdc96c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:13.126550 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792ddaa-a834-4679-8d15-e88359fdc96c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:13.131132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:13.132024 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:13.132307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:13.132583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] PATH_INFO: `/v3/users/cee061b2f23f4e87b472680d87375e80` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:13.133774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee061b2f23f4e87b472680d87375e80)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:13.135739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:13.163578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:13.183897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee061b2f23f4e87b472680d87375e80'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:13.184555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee061b2f23f4e87b472680d87375e80'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:13.191938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee061b2f23f4e87b472680d87375e80'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:13.203739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee061b2f23f4e87b472680d87375e80'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:13.211064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee061b2f23f4e87b472680d87375e80'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:13.216979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1057/2126] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:03:13 2022] DELETE /identity/v3/users/cee061b2f23f4e87b472680d87375e80 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:13.221456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e7680-3d15-4fc8-95a4-3c271751dfc5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:13.221744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e7680-3d15-4fc8-95a4-3c271751dfc5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:13.222024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e7680-3d15-4fc8-95a4-3c271751dfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:13.239152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-426e7680-3d15-4fc8-95a4-3c271751dfc5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:13.240379 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-426e7680-3d15-4fc8-95a4-3c271751dfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:13.300796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1070/2127] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:13.311272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:13.312693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:13.374494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfaf75-6df4-4816-bc44-a5371898f615 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:13.378577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b3890-65f4-4f68-9f61-c07e7d941b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:13.379452 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b3890-65f4-4f68-9f61-c07e7d941b52 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:13.379838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b3890-65f4-4f68-9f61-c07e7d941b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:13.380700 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b3890-65f4-4f68-9f61-c07e7d941b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:13.391356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b3890-65f4-4f68-9f61-c07e7d941b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:13.392956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b3890-65f4-4f68-9f61-c07e7d941b52 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:13.407305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1058/2128] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:13.679202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e7680-3d15-4fc8-95a4-3c271751dfc5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:13.681062 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-426e7680-3d15-4fc8-95a4-3c271751dfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:13.694671 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e7680-3d15-4fc8-95a4-3c271751dfc5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:13.699206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:13.700110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:13.700392 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:13.700671 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66 admin admin] PATH_INFO: `/v3/projects/361c479558b7471ca4b92a7727dee1cf` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:13.701909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=361c479558b7471ca4b92a7727dee1cf)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:13.703951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:13.751059 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '361c479558b7471ca4b92a7727dee1cf', 'request_id': 'req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:13.757581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '361c479558b7471ca4b92a7727dee1cf', 'request_id': 'req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:13.763162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '361c479558b7471ca4b92a7727dee1cf', 'request_id': 'req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:13.775591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66 admin admin] The token cache is being invalidate because project 361c479558b7471ca4b92a7727dee1cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:03:13.775898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:13.776506 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c2fca7aa-78f2-4b1a-b9d1-37bdcc0ebe66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:13.778023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1071/2129] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:03:13 2022] DELETE /identity/v3/projects/361c479558b7471ca4b92a7727dee1cf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:23.402533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=6,keystone:SELECT=45,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:23.772089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=83,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:28.596632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b3890-65f4-4f68-9f61-c07e7d941b52 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:28.598705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9a2b3890-65f4-4f68-9f61-c07e7d941b52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:28.614645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b3890-65f4-4f68-9f61-c07e7d941b52 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:28.681414 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:28.682462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:28.682748 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:28.683025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] PATH_INFO: `/v3/users/f2b2bb8167764cf48f42d0b466a1b6b5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:28.684589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2b2bb8167764cf48f42d0b466a1b6b5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:28.686556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:28.734876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2b2bb8167764cf48f42d0b466a1b6b5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:28.735536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2b2bb8167764cf48f42d0b466a1b6b5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:28.742771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2b2bb8167764cf48f42d0b466a1b6b5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:28.755760 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2b2bb8167764cf48f42d0b466a1b6b5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:28.763750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2b2bb8167764cf48f42d0b466a1b6b5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:28.769894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1059/2130] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:03:28 2022] DELETE /identity/v3/users/f2b2bb8167764cf48f42d0b466a1b6b5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:28.774916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6f863-e93f-4b14-9c8e-1773396a3ba0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:28.775373 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6f863-e93f-4b14-9c8e-1773396a3ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:28.775462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6f863-e93f-4b14-9c8e-1773396a3ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:28.793562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e6d6f863-e93f-4b14-9c8e-1773396a3ba0 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:28.794865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e6d6f863-e93f-4b14-9c8e-1773396a3ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:28.886450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1072/2131] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:28.910426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:28.912436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:28.977516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5cb7c-2874-46d3-9819-8903a7aceb48 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:28.981130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7315fa9e-8946-4904-827c-34c4385480c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:28.981781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315fa9e-8946-4904-827c-34c4385480c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:28.982064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315fa9e-8946-4904-827c-34c4385480c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:28.982339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315fa9e-8946-4904-827c-34c4385480c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:28.992183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7315fa9e-8946-4904-827c-34c4385480c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:28.994270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7315fa9e-8946-4904-827c-34c4385480c1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:29.005329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1060/2132] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:29.301217 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6f863-e93f-4b14-9c8e-1773396a3ba0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:29.302666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e6d6f863-e93f-4b14-9c8e-1773396a3ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:29.317246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6f863-e93f-4b14-9c8e-1773396a3ba0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:29.322525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27992ed-0e40-4527-b0ad-4d83327e4a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:29.323399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27992ed-0e40-4527-b0ad-4d83327e4a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:29.323705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27992ed-0e40-4527-b0ad-4d83327e4a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:29.323988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27992ed-0e40-4527-b0ad-4d83327e4a23 admin admin] PATH_INFO: `/v3/projects/4b7f8a3cc4db4355ae2093b4b9b5199d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:29.326800 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27992ed-0e40-4527-b0ad-4d83327e4a23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b7f8a3cc4db4355ae2093b4b9b5199d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:29.328969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27992ed-0e40-4527-b0ad-4d83327e4a23 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:29.379359 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e27992ed-0e40-4527-b0ad-4d83327e4a23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b7f8a3cc4db4355ae2093b4b9b5199d', 'request_id': 'req-e27992ed-0e40-4527-b0ad-4d83327e4a23'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:29.387712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e27992ed-0e40-4527-b0ad-4d83327e4a23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b7f8a3cc4db4355ae2093b4b9b5199d', 'request_id': 'req-e27992ed-0e40-4527-b0ad-4d83327e4a23'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:29.393665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e27992ed-0e40-4527-b0ad-4d83327e4a23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b7f8a3cc4db4355ae2093b4b9b5199d', 'request_id': 'req-e27992ed-0e40-4527-b0ad-4d83327e4a23'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:29.407228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e27992ed-0e40-4527-b0ad-4d83327e4a23 admin admin] The token cache is being invalidate because project 4b7f8a3cc4db4355ae2093b4b9b5199d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:03:29.408896 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e27992ed-0e40-4527-b0ad-4d83327e4a23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:29.408896 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e27992ed-0e40-4527-b0ad-4d83327e4a23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:29.410163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1073/2133] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:03:29 2022] DELETE /identity/v3/projects/4b7f8a3cc4db4355ae2093b4b9b5199d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:29.415093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6ac90-ec54-41b0-bfea-2c24c3d8c808 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:29.415601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6ac90-ec54-41b0-bfea-2c24c3d8c808 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:29.415911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6ac90-ec54-41b0-bfea-2c24c3d8c808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:29.435524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7bf6ac90-ec54-41b0-bfea-2c24c3d8c808 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:29.436847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7bf6ac90-ec54-41b0-bfea-2c24c3d8c808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:29.460295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:29.507288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1061/2134] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:03:29.518192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735853c-da09-4a38-af04-8f76f7b27b5c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:29.518192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735853c-da09-4a38-af04-8f76f7b27b5c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:29.518192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735853c-da09-4a38-af04-8f76f7b27b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:29.539581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0735853c-da09-4a38-af04-8f76f7b27b5c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:29.540849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0735853c-da09-4a38-af04-8f76f7b27b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:29.555959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1074/2135] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:29.560248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6ac90-ec54-41b0-bfea-2c24c3d8c808 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:29.573820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6ac90-ec54-41b0-bfea-2c24c3d8c808 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:29.578438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc330a6c-fdf9-4317-b625-a8250a77a0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:29.579305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc330a6c-fdf9-4317-b625-a8250a77a0ec admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:29.579593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc330a6c-fdf9-4317-b625-a8250a77a0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:29.579899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc330a6c-fdf9-4317-b625-a8250a77a0ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:29.580635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc330a6c-fdf9-4317-b625-a8250a77a0ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:03:29.580941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc330a6c-fdf9-4317-b625-a8250a77a0ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:29.582568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc330a6c-fdf9-4317-b625-a8250a77a0ec admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:29.594688 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1062/2136] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:03:29.599095 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735853c-da09-4a38-af04-8f76f7b27b5c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:29.609376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735853c-da09-4a38-af04-8f76f7b27b5c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:29.614123 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688157f-f07c-4361-b05e-61bae48e03c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:29.614783 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688157f-f07c-4361-b05e-61bae48e03c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:29.615064 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688157f-f07c-4361-b05e-61bae48e03c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:29.615340 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688157f-f07c-4361-b05e-61bae48e03c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:29.616075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688157f-f07c-4361-b05e-61bae48e03c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:29.618531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688157f-f07c-4361-b05e-61bae48e03c7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:29.650738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1075/2137] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:03:29 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:29.654827 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc330a6c-fdf9-4317-b625-a8250a77a0ec admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:29.665023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc330a6c-fdf9-4317-b625-a8250a77a0ec admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:29.738461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c22773-3632-4685-a7a3-2dec9306a2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:29.739378 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c22773-3632-4685-a7a3-2dec9306a2ee admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:29.739880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c22773-3632-4685-a7a3-2dec9306a2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:29.740362 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c22773-3632-4685-a7a3-2dec9306a2ee admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:29.741299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c22773-3632-4685-a7a3-2dec9306a2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:29.743537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c22773-3632-4685-a7a3-2dec9306a2ee admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:29.775527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1063/2138] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:03:29 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:29.776814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688157f-f07c-4361-b05e-61bae48e03c7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:29.788130 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688157f-f07c-4361-b05e-61bae48e03c7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:29.793664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6c04d-fea2-4fa1-98c1-95973cfbf175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:29.794578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6c04d-fea2-4fa1-98c1-95973cfbf175 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:29.795058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6c04d-fea2-4fa1-98c1-95973cfbf175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:29.795404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6c04d-fea2-4fa1-98c1-95973cfbf175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:29.796450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6c04d-fea2-4fa1-98c1-95973cfbf175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:29.798756 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6c04d-fea2-4fa1-98c1-95973cfbf175 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:29.814683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1076/2139] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:03:29.815677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c22773-3632-4685-a7a3-2dec9306a2ee admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:29.826814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c22773-3632-4685-a7a3-2dec9306a2ee admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:29.831732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012f093-8899-43fd-bf81-bcb8bce9eae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:29.832420 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012f093-8899-43fd-bf81-bcb8bce9eae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:29.832994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012f093-8899-43fd-bf81-bcb8bce9eae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:29.833282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012f093-8899-43fd-bf81-bcb8bce9eae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:29.834244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012f093-8899-43fd-bf81-bcb8bce9eae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:29.836137 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012f093-8899-43fd-bf81-bcb8bce9eae9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:29.847293 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1064/2140] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:03:29.851279 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6c04d-fea2-4fa1-98c1-95973cfbf175 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:29.861187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6c04d-fea2-4fa1-98c1-95973cfbf175 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:29.865765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7a584-8f70-460a-b222-40b9ecbbdd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:29.866654 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7a584-8f70-460a-b222-40b9ecbbdd36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:29.866915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7a584-8f70-460a-b222-40b9ecbbdd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:29.867187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7a584-8f70-460a-b222-40b9ecbbdd36 admin admin] PATH_INFO: `/v3/projects/110edda657dd4b0ea11456722f988a4f/users/d54c5aeceff14f70a2d4b2c95252dad9/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:29.886447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7a584-8f70-460a-b222-40b9ecbbdd36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=110edda657dd4b0ea11456722f988a4f, user_id=d54c5aeceff14f70a2d4b2c95252dad9, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:29.889149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7a584-8f70-460a-b222-40b9ecbbdd36 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:29.902986 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1077/2141] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:03:29 2022] PUT /identity/v3/projects/110edda657dd4b0ea11456722f988a4f/users/d54c5aeceff14f70a2d4b2c95252dad9/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:29.914804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53becb17-99c9-4887-b331-979dba3d2487 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:29.915127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53becb17-99c9-4887-b331-979dba3d2487 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:29.915407 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53becb17-99c9-4887-b331-979dba3d2487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:29.945522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-53becb17-99c9-4887-b331-979dba3d2487 None None] MFA Rules not processed for user `d54c5aeceff14f70a2d4b2c95252dad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:29.946761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-53becb17-99c9-4887-b331-979dba3d2487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:30.015013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1065/2142] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:30.026546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7a584-8f70-460a-b222-40b9ecbbdd36 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:30.028200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-26f7a584-8f70-460a-b222-40b9ecbbdd36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:30.051023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:30.102678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7a584-8f70-460a-b222-40b9ecbbdd36 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:30.107037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b9838-9a0f-499b-9e78-ebdaa3a69ee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:30.107835 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b9838-9a0f-499b-9e78-ebdaa3a69ee5 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:30.108184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b9838-9a0f-499b-9e78-ebdaa3a69ee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:30.108649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b9838-9a0f-499b-9e78-ebdaa3a69ee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:30.118635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b9838-9a0f-499b-9e78-ebdaa3a69ee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:30.120111 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b9838-9a0f-499b-9e78-ebdaa3a69ee5 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:30.129711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1078/2143] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:33.021789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4e342-2ac1-4902-b4f6-d8c15e00d161 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.022275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4e342-2ac1-4902-b4f6-d8c15e00d161 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.022368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4e342-2ac1-4902-b4f6-d8c15e00d161 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:33.023868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1066/2144] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 11:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:33.033145 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb1ba5-2b65-4584-845a-f9196434ffe4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.033618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb1ba5-2b65-4584-845a-f9196434ffe4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.033718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb1ba5-2b65-4584-845a-f9196434ffe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:33.051949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5dbb1ba5-2b65-4584-845a-f9196434ffe4 None None] MFA Rules not processed for user `18fc0e7956b14a49beab025922e8ff81`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:33.053015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5dbb1ba5-2b65-4584-845a-f9196434ffe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:33.109842 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4e342-2ac1-4902-b4f6-d8c15e00d161 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:33.112928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-80a4e342-2ac1-4902-b4f6-d8c15e00d161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:33.129165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1079/2145] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:33.136682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:33.140456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb1ba5-2b65-4584-845a-f9196434ffe4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:33.141901 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5dbb1ba5-2b65-4584-845a-f9196434ffe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:33.154370 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb1ba5-2b65-4584-845a-f9196434ffe4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:33.160090 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdeb0aa-29ec-45d2-ab1d-9067f6de7a4b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:33.160877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdeb0aa-29ec-45d2-ab1d-9067f6de7a4b service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.161163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdeb0aa-29ec-45d2-ab1d-9067f6de7a4b service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.161440 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdeb0aa-29ec-45d2-ab1d-9067f6de7a4b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:33.170619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdeb0aa-29ec-45d2-ab1d-9067f6de7a4b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:33.172231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdeb0aa-29ec-45d2-ab1d-9067f6de7a4b service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:33.182425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1080/2146] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:03:33.186258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4e342-2ac1-4902-b4f6-d8c15e00d161 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:33.190862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d1923-a26d-466f-b46e-5ef837ff36c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:33.191754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d1923-a26d-466f-b46e-5ef837ff36c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.192038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d1923-a26d-466f-b46e-5ef837ff36c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.192308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d1923-a26d-466f-b46e-5ef837ff36c1 admin admin] PATH_INFO: `/v3/users/f6390dc942db430e8dd648901900432b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:33.193518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d1923-a26d-466f-b46e-5ef837ff36c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6390dc942db430e8dd648901900432b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:33.197594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d1923-a26d-466f-b46e-5ef837ff36c1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:33.245200 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ef5d1923-a26d-466f-b46e-5ef837ff36c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6390dc942db430e8dd648901900432b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:33.245800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ef5d1923-a26d-466f-b46e-5ef837ff36c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6390dc942db430e8dd648901900432b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:33.252927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ef5d1923-a26d-466f-b46e-5ef837ff36c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6390dc942db430e8dd648901900432b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:33.265509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ef5d1923-a26d-466f-b46e-5ef837ff36c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6390dc942db430e8dd648901900432b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:33.272404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ef5d1923-a26d-466f-b46e-5ef837ff36c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6390dc942db430e8dd648901900432b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:33.278879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1067/2147] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:03:33 2022] DELETE /identity/v3/users/f6390dc942db430e8dd648901900432b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:03:33.451603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdeb0aa-29ec-45d2-ab1d-9067f6de7a4b service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:33.464643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdeb0aa-29ec-45d2-ab1d-9067f6de7a4b service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:33.522129 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d03ac-c913-431d-b2b5-6ee5043ee430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:33.522994 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d03ac-c913-431d-b2b5-6ee5043ee430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.523280 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d03ac-c913-431d-b2b5-6ee5043ee430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.523558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d03ac-c913-431d-b2b5-6ee5043ee430 admin admin] PATH_INFO: `/v3/projects/c6c6487f3e094808b1665bdd93da9b3c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:33.524776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d03ac-c913-431d-b2b5-6ee5043ee430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6c6487f3e094808b1665bdd93da9b3c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:33.526733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d03ac-c913-431d-b2b5-6ee5043ee430 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:33.570175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:33.572260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cb7d03ac-c913-431d-b2b5-6ee5043ee430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6c6487f3e094808b1665bdd93da9b3c', 'request_id': 'req-cb7d03ac-c913-431d-b2b5-6ee5043ee430'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:33.579708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cb7d03ac-c913-431d-b2b5-6ee5043ee430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6c6487f3e094808b1665bdd93da9b3c', 'request_id': 'req-cb7d03ac-c913-431d-b2b5-6ee5043ee430'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:33.585169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cb7d03ac-c913-431d-b2b5-6ee5043ee430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6c6487f3e094808b1665bdd93da9b3c', 'request_id': 'req-cb7d03ac-c913-431d-b2b5-6ee5043ee430'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:33.597685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cb7d03ac-c913-431d-b2b5-6ee5043ee430 admin admin] The token cache is being invalidate because project c6c6487f3e094808b1665bdd93da9b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:03:33.598019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cb7d03ac-c913-431d-b2b5-6ee5043ee430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:33.598605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-cb7d03ac-c913-431d-b2b5-6ee5043ee430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:33.600165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1081/2148] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:03:33 2022] DELETE /identity/v3/projects/c6c6487f3e094808b1665bdd93da9b3c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:33.605365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583154cc-7b66-478d-908c-b00f18a24320 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.605655 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583154cc-7b66-478d-908c-b00f18a24320 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.605934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583154cc-7b66-478d-908c-b00f18a24320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:33.624719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-583154cc-7b66-478d-908c-b00f18a24320 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:33.626451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-583154cc-7b66-478d-908c-b00f18a24320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:33.685565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1068/2149] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:33.693257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa5eda-211e-45a9-b1d2-000f05283063 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.693542 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa5eda-211e-45a9-b1d2-000f05283063 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.693816 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa5eda-211e-45a9-b1d2-000f05283063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:33.712604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-10aa5eda-211e-45a9-b1d2-000f05283063 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:33.713791 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-10aa5eda-211e-45a9-b1d2-000f05283063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:33.721513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1082/2150] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:33.725554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583154cc-7b66-478d-908c-b00f18a24320 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:33.735376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583154cc-7b66-478d-908c-b00f18a24320 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:33.739973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d812328-a89f-4491-b7fb-432f4273bd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:33.740626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d812328-a89f-4491-b7fb-432f4273bd52 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.740909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d812328-a89f-4491-b7fb-432f4273bd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.741186 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d812328-a89f-4491-b7fb-432f4273bd52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:33.741917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d812328-a89f-4491-b7fb-432f4273bd52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:03:33.742229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d812328-a89f-4491-b7fb-432f4273bd52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:33.743841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d812328-a89f-4491-b7fb-432f4273bd52 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:33.759831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1069/2151] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:03:33.760939 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa5eda-211e-45a9-b1d2-000f05283063 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:33.770534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa5eda-211e-45a9-b1d2-000f05283063 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:33.776031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56735228-3ef1-4ed4-8554-82e99f1255c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:33.776682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56735228-3ef1-4ed4-8554-82e99f1255c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.776964 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56735228-3ef1-4ed4-8554-82e99f1255c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.777242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56735228-3ef1-4ed4-8554-82e99f1255c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:33.777948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56735228-3ef1-4ed4-8554-82e99f1255c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:33.779869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56735228-3ef1-4ed4-8554-82e99f1255c8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:33.805525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1083/2152] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:03:33 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:03:33.809607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d812328-a89f-4491-b7fb-432f4273bd52 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:33.819570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d812328-a89f-4491-b7fb-432f4273bd52 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:33.840917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:33.879216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834afd59-50c5-4744-a5e3-b439f388d289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:33.879916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834afd59-50c5-4744-a5e3-b439f388d289 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.880208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834afd59-50c5-4744-a5e3-b439f388d289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.880492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834afd59-50c5-4744-a5e3-b439f388d289 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:33.881193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834afd59-50c5-4744-a5e3-b439f388d289 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:33.883691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834afd59-50c5-4744-a5e3-b439f388d289 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:33.903326 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1070/2153] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:03:33 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:03:33.907503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56735228-3ef1-4ed4-8554-82e99f1255c8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:33.918067 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56735228-3ef1-4ed4-8554-82e99f1255c8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:33.922758 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344cfe0-c4e7-43f4-a122-354173474d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:33.923420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344cfe0-c4e7-43f4-a122-354173474d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.923739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344cfe0-c4e7-43f4-a122-354173474d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.924021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344cfe0-c4e7-43f4-a122-354173474d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:33.924769 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344cfe0-c4e7-43f4-a122-354173474d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:33.926338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344cfe0-c4e7-43f4-a122-354173474d2f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:33.937012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1084/2154] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:03:33.941532 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834afd59-50c5-4744-a5e3-b439f388d289 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:33.954806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834afd59-50c5-4744-a5e3-b439f388d289 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:33.959620 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062eca9-2bd0-4523-87af-090a6bb3da79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:33.960323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062eca9-2bd0-4523-87af-090a6bb3da79 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.960611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062eca9-2bd0-4523-87af-090a6bb3da79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.960965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062eca9-2bd0-4523-87af-090a6bb3da79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:33.962192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062eca9-2bd0-4523-87af-090a6bb3da79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:33.963806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062eca9-2bd0-4523-87af-090a6bb3da79 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:33.974283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1071/2155] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:03:33.978546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344cfe0-c4e7-43f4-a122-354173474d2f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:33.988595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344cfe0-c4e7-43f4-a122-354173474d2f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:33.993458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b83877-d027-4eb8-a547-37373b87deca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:33.994460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b83877-d027-4eb8-a547-37373b87deca admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:33.994753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b83877-d027-4eb8-a547-37373b87deca admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:33.995042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b83877-d027-4eb8-a547-37373b87deca admin admin] PATH_INFO: `/v3/projects/172a9e51095e47318b27881dab868355/users/94bcc16975124f728459d3c28b97f1a8/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:34.011228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b83877-d027-4eb8-a547-37373b87deca admin admin] RBAC: Authorizing `identity:create_grant(project_id=172a9e51095e47318b27881dab868355, user_id=94bcc16975124f728459d3c28b97f1a8, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:34.013942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b83877-d027-4eb8-a547-37373b87deca admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:34.028559 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1085/2156] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:03:33 2022] PUT /identity/v3/projects/172a9e51095e47318b27881dab868355/users/94bcc16975124f728459d3c28b97f1a8/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:03:34.033940 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2357db-4653-42c9-ba91-7f2257ab82b1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:34.034451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2357db-4653-42c9-ba91-7f2257ab82b1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:34.034556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2357db-4653-42c9-ba91-7f2257ab82b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:34.051806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1c2357db-4653-42c9-ba91-7f2257ab82b1 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:34.053010 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1c2357db-4653-42c9-ba91-7f2257ab82b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:34.122509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1072/2157] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:34.131942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b83877-d027-4eb8-a547-37373b87deca admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:34.134519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-34b83877-d027-4eb8-a547-37373b87deca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:34.191753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b83877-d027-4eb8-a547-37373b87deca admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:34.196009 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf537e-5d94-4273-afcf-54fd11975180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:34.196667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf537e-5d94-4273-afcf-54fd11975180 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:34.196955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf537e-5d94-4273-afcf-54fd11975180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:34.197235 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf537e-5d94-4273-afcf-54fd11975180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:34.206172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf537e-5d94-4273-afcf-54fd11975180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:34.207597 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf537e-5d94-4273-afcf-54fd11975180 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:34.217025 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1086/2158] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:38.514234 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2357db-4653-42c9-ba91-7f2257ab82b1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:38.515768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1c2357db-4653-42c9-ba91-7f2257ab82b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:38.531988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2357db-4653-42c9-ba91-7f2257ab82b1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:38.539806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0eec38-209e-424b-9a56-35dcc83e547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:38.540768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0eec38-209e-424b-9a56-35dcc83e547b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:38.541226 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0eec38-209e-424b-9a56-35dcc83e547b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:38.541564 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0eec38-209e-424b-9a56-35dcc83e547b admin admin] PATH_INFO: `/v3/users/d54c5aeceff14f70a2d4b2c95252dad9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:38.545297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0eec38-209e-424b-9a56-35dcc83e547b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d54c5aeceff14f70a2d4b2c95252dad9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:38.547583 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0eec38-209e-424b-9a56-35dcc83e547b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:38.644892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a0eec38-209e-424b-9a56-35dcc83e547b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd54c5aeceff14f70a2d4b2c95252dad9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:38.646023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a0eec38-209e-424b-9a56-35dcc83e547b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd54c5aeceff14f70a2d4b2c95252dad9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:38.657864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a0eec38-209e-424b-9a56-35dcc83e547b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd54c5aeceff14f70a2d4b2c95252dad9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:38.678313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a0eec38-209e-424b-9a56-35dcc83e547b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd54c5aeceff14f70a2d4b2c95252dad9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:38.694441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a0eec38-209e-424b-9a56-35dcc83e547b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd54c5aeceff14f70a2d4b2c95252dad9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:38.703179 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1073/2159] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:03:38 2022] DELETE /identity/v3/users/d54c5aeceff14f70a2d4b2c95252dad9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:38.707500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc0508-68d5-4d3a-971b-a0cd9440d1e8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:38.708415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc0508-68d5-4d3a-971b-a0cd9440d1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:38.709121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc0508-68d5-4d3a-971b-a0cd9440d1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:38.739184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2df11e-66c9-468d-8a7f-914262e0a153 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:38.739580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2df11e-66c9-468d-8a7f-914262e0a153 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:38.740126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2df11e-66c9-468d-8a7f-914262e0a153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:38.742980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-66cc0508-68d5-4d3a-971b-a0cd9440d1e8 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:38.744228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-66cc0508-68d5-4d3a-971b-a0cd9440d1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:38.785535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ac2df11e-66c9-468d-8a7f-914262e0a153 None None] MFA Rules not processed for user `94bcc16975124f728459d3c28b97f1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:38.787062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ac2df11e-66c9-468d-8a7f-914262e0a153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:38.796904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:38.832157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1087/2160] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:03:38.839482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc0508-68d5-4d3a-971b-a0cd9440d1e8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:38.841286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-66cc0508-68d5-4d3a-971b-a0cd9440d1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:38.888228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:38.935213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1074/2161] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:38.954444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc0508-68d5-4d3a-971b-a0cd9440d1e8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:38.958702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1753d-0e4b-4120-b2c7-207d4352f08c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:38.959590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1753d-0e4b-4120-b2c7-207d4352f08c service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:38.960172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1753d-0e4b-4120-b2c7-207d4352f08c service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:38.960634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1753d-0e4b-4120-b2c7-207d4352f08c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:38.971467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1753d-0e4b-4120-b2c7-207d4352f08c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:38.973152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1753d-0e4b-4120-b2c7-207d4352f08c service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:38.985263 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1088/2162] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:39.071406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2df11e-66c9-468d-8a7f-914262e0a153 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:39.074148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ac2df11e-66c9-468d-8a7f-914262e0a153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:39.194387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2df11e-66c9-468d-8a7f-914262e0a153 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:39.201475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f42edf-6807-4f96-9d02-06155dfe4ec8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:39.202138 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f42edf-6807-4f96-9d02-06155dfe4ec8 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.202546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f42edf-6807-4f96-9d02-06155dfe4ec8 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.202837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f42edf-6807-4f96-9d02-06155dfe4ec8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.212910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f42edf-6807-4f96-9d02-06155dfe4ec8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:39.214689 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f42edf-6807-4f96-9d02-06155dfe4ec8 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:39.224723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1075/2163] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:39.340739 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1753d-0e4b-4120-b2c7-207d4352f08c service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:39.342379 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-61d1753d-0e4b-4120-b2c7-207d4352f08c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:39.357758 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1753d-0e4b-4120-b2c7-207d4352f08c service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:39.363291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:39.364478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.364845 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.365411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889 admin admin] PATH_INFO: `/v3/projects/110edda657dd4b0ea11456722f988a4f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.366872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=110edda657dd4b0ea11456722f988a4f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:39.369464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:39.426821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '110edda657dd4b0ea11456722f988a4f', 'request_id': 'req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:39.434116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '110edda657dd4b0ea11456722f988a4f', 'request_id': 'req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:39.440193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '110edda657dd4b0ea11456722f988a4f', 'request_id': 'req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:39.455058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889 admin admin] The token cache is being invalidate because project 110edda657dd4b0ea11456722f988a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:03:39.455439 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:39.456186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fcd4e5b7-6aa9-4d7b-9585-52b70218a889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:03:39.458095 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1089/2164] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:03:39 2022] DELETE /identity/v3/projects/110edda657dd4b0ea11456722f988a4f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:39.463644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05421e0-faa4-48ff-a886-f646f900da15 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.464095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05421e0-faa4-48ff-a886-f646f900da15 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.464498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05421e0-faa4-48ff-a886-f646f900da15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.484553 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b05421e0-faa4-48ff-a886-f646f900da15 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:39.485214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b05421e0-faa4-48ff-a886-f646f900da15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:39.553721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1076/2165] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:39.562182 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6b48c-02d4-415b-a487-791f963ff3bb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.562502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6b48c-02d4-415b-a487-791f963ff3bb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.562786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6b48c-02d4-415b-a487-791f963ff3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.584283 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-40c6b48c-02d4-415b-a487-791f963ff3bb None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:39.585487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-40c6b48c-02d4-415b-a487-791f963ff3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:39.593778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1090/2166] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:39.597919 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05421e0-faa4-48ff-a886-f646f900da15 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:39.608795 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05421e0-faa4-48ff-a886-f646f900da15 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:39.615352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407215e4-d148-4ccf-9f44-2ccf4ddb5426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:39.615352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407215e4-d148-4ccf-9f44-2ccf4ddb5426 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.615352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407215e4-d148-4ccf-9f44-2ccf4ddb5426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.615732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407215e4-d148-4ccf-9f44-2ccf4ddb5426 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.616470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407215e4-d148-4ccf-9f44-2ccf4ddb5426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:03:39.616821 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407215e4-d148-4ccf-9f44-2ccf4ddb5426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:39.619465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407215e4-d148-4ccf-9f44-2ccf4ddb5426 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:39.634063 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1077/2167] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:03:39.638124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6b48c-02d4-415b-a487-791f963ff3bb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:39.648984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6b48c-02d4-415b-a487-791f963ff3bb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:39.653710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1379f-3fa7-4b01-83ba-e75f272d65d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:39.654385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1379f-3fa7-4b01-83ba-e75f272d65d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.654667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1379f-3fa7-4b01-83ba-e75f272d65d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.654938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1379f-3fa7-4b01-83ba-e75f272d65d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.655632 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1379f-3fa7-4b01-83ba-e75f272d65d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:39.658318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1379f-3fa7-4b01-83ba-e75f272d65d1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:39.696389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1091/2168] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:03:39 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:39.701020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407215e4-d148-4ccf-9f44-2ccf4ddb5426 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:39.722001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407215e4-d148-4ccf-9f44-2ccf4ddb5426 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:39.738121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:39.805572 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169208bb-46cd-4f0c-98ae-7dd5524550b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:39.806361 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169208bb-46cd-4f0c-98ae-7dd5524550b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.806759 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169208bb-46cd-4f0c-98ae-7dd5524550b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.807040 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169208bb-46cd-4f0c-98ae-7dd5524550b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.807853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169208bb-46cd-4f0c-98ae-7dd5524550b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:39.809928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169208bb-46cd-4f0c-98ae-7dd5524550b7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:39.834187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1078/2169] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:03:39 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:39.838140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1379f-3fa7-4b01-83ba-e75f272d65d1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:39.854655 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1379f-3fa7-4b01-83ba-e75f272d65d1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:39.862916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8a972-b410-4152-88f8-cee7cc53e85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:39.863566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8a972-b410-4152-88f8-cee7cc53e85d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.863883 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8a972-b410-4152-88f8-cee7cc53e85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.864167 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8a972-b410-4152-88f8-cee7cc53e85d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.864908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8a972-b410-4152-88f8-cee7cc53e85d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:39.866548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8a972-b410-4152-88f8-cee7cc53e85d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:39.880253 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1092/2170] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:03:39.884265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169208bb-46cd-4f0c-98ae-7dd5524550b7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:39.894962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169208bb-46cd-4f0c-98ae-7dd5524550b7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:39.900087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976f63e-0f83-4edb-9f00-db861b4d6445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:39.900977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976f63e-0f83-4edb-9f00-db861b4d6445 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.901371 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976f63e-0f83-4edb-9f00-db861b4d6445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.901666 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976f63e-0f83-4edb-9f00-db861b4d6445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.902439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976f63e-0f83-4edb-9f00-db861b4d6445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:39.904106 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976f63e-0f83-4edb-9f00-db861b4d6445 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:39.915418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1079/2171] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:03:39.919489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8a972-b410-4152-88f8-cee7cc53e85d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:39.929404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8a972-b410-4152-88f8-cee7cc53e85d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:39.934644 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa36a00-69c3-4c3a-ac35-202bb39ad728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:39.935525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa36a00-69c3-4c3a-ac35-202bb39ad728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.935849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa36a00-69c3-4c3a-ac35-202bb39ad728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.936135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa36a00-69c3-4c3a-ac35-202bb39ad728 admin admin] PATH_INFO: `/v3/projects/1fccc592269f433b99c2d13dad5e8e39/users/8f8da2142b9e4751b436cfd4920587a0/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.956336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa36a00-69c3-4c3a-ac35-202bb39ad728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fccc592269f433b99c2d13dad5e8e39, user_id=8f8da2142b9e4751b436cfd4920587a0, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:39.959041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa36a00-69c3-4c3a-ac35-202bb39ad728 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:39.974724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1093/2172] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:03:39 2022] PUT /identity/v3/projects/1fccc592269f433b99c2d13dad5e8e39/users/8f8da2142b9e4751b436cfd4920587a0/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:03:39.980009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b4a91-4eb3-44ef-9f66-92bd1d1f36d9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.980304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b4a91-4eb3-44ef-9f66-92bd1d1f36d9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.980591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b4a91-4eb3-44ef-9f66-92bd1d1f36d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.999841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-229b4a91-4eb3-44ef-9f66-92bd1d1f36d9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:40.001143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-229b4a91-4eb3-44ef-9f66-92bd1d1f36d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:40.107445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1080/2173] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:40.135631 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa36a00-69c3-4c3a-ac35-202bb39ad728 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:40.138081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3aa36a00-69c3-4c3a-ac35-202bb39ad728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:40.187827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:40.221728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa36a00-69c3-4c3a-ac35-202bb39ad728 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:40.226321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc3c9e-ada0-487d-8bad-50e2c4f82812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:40.227777 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc3c9e-ada0-487d-8bad-50e2c4f82812 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:40.228928 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc3c9e-ada0-487d-8bad-50e2c4f82812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:40.229514 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc3c9e-ada0-487d-8bad-50e2c4f82812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:40.241419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc3c9e-ada0-487d-8bad-50e2c4f82812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:40.243390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc3c9e-ada0-487d-8bad-50e2c4f82812 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:40.276718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1094/2174] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:42.102153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9142-6555-4041-bf7b-25c9a5867d1a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:42.102731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9142-6555-4041-bf7b-25c9a5867d1a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:42.103214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9142-6555-4041-bf7b-25c9a5867d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:42.143534 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f9ba9142-6555-4041-bf7b-25c9a5867d1a None None] MFA Rules not processed for user `94bcc16975124f728459d3c28b97f1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:42.144989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f9ba9142-6555-4041-bf7b-25c9a5867d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:42.237775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1081/2175] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:43.317698 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee6b86-2ecd-45ab-a602-8f31feab5d99 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:43.318192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee6b86-2ecd-45ab-a602-8f31feab5d99 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:43.318284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee6b86-2ecd-45ab-a602-8f31feab5d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:43.356236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f6ee6b86-2ecd-45ab-a602-8f31feab5d99 None None] MFA Rules not processed for user `17071129652c4fa09b3b4b20c0dc0725`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:43.357577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f6ee6b86-2ecd-45ab-a602-8f31feab5d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:43.499065 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1095/2176] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:47.878532 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ef946-ba53-4617-bc57-203aca8e0388 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:47.879055 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ef946-ba53-4617-bc57-203aca8e0388 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:47.879150 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ef946-ba53-4617-bc57-203aca8e0388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:47.909835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-097ef946-ba53-4617-bc57-203aca8e0388 None None] MFA Rules not processed for user `8f8da2142b9e4751b436cfd4920587a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:47.911064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-097ef946-ba53-4617-bc57-203aca8e0388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:47.946563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:47.986631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1082/2177] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:47.991190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee6b86-2ecd-45ab-a602-8f31feab5d99 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:48.003924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee6b86-2ecd-45ab-a602-8f31feab5d99 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:48.010684 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2427c7-8e6d-44fe-b171-0a368b2de8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:48.011579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2427c7-8e6d-44fe-b171-0a368b2de8a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:48.011893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2427c7-8e6d-44fe-b171-0a368b2de8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:48.012177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2427c7-8e6d-44fe-b171-0a368b2de8a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:48.012876 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2427c7-8e6d-44fe-b171-0a368b2de8a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:48.015559 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2427c7-8e6d-44fe-b171-0a368b2de8a1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:48.051886 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1096/2178] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:03:47 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:48.057082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ef946-ba53-4617-bc57-203aca8e0388 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:48.071376 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ef946-ba53-4617-bc57-203aca8e0388 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:48.239122 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff83d3b-e254-48e2-bea4-96009580c0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:48.240212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff83d3b-e254-48e2-bea4-96009580c0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:48.240694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff83d3b-e254-48e2-bea4-96009580c0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:48.241129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff83d3b-e254-48e2-bea4-96009580c0a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:48.241984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff83d3b-e254-48e2-bea4-96009580c0a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:48.246958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff83d3b-e254-48e2-bea4-96009580c0a2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:48.300343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1083/2179] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:03:48 2022] POST /identity/v3/users => generated 296 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:03:48.304312 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2427c7-8e6d-44fe-b171-0a368b2de8a1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:48.325715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2427c7-8e6d-44fe-b171-0a368b2de8a1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:48.331915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515b788-5d00-4df0-82c2-a7539e8964e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:48.332952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515b788-5d00-4df0-82c2-a7539e8964e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:48.333445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515b788-5d00-4df0-82c2-a7539e8964e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:48.334590 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515b788-5d00-4df0-82c2-a7539e8964e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:48.336511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515b788-5d00-4df0-82c2-a7539e8964e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:48.339159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515b788-5d00-4df0-82c2-a7539e8964e4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:48.360360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff83d3b-e254-48e2-bea4-96009580c0a2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:48.371510 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1097/2180] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:03:48.376373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff83d3b-e254-48e2-bea4-96009580c0a2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:48.397044 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1a83c-ba4e-475f-abef-92388f363d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:48.397992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1a83c-ba4e-475f-abef-92388f363d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:48.398340 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1a83c-ba4e-475f-abef-92388f363d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:48.398547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1a83c-ba4e-475f-abef-92388f363d5f admin admin] PATH_INFO: `/v3/projects/80e78c5ecac645ffbc1a9dbc3aa0f82e/users/4e41807dbd8a4922ba199f4bab63a9b5/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:48.417362 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a1a83c-ba4e-475f-abef-92388f363d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e78c5ecac645ffbc1a9dbc3aa0f82e, user_id=4e41807dbd8a4922ba199f4bab63a9b5, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:48.419293 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a1a83c-ba4e-475f-abef-92388f363d5f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:48.450017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515b788-5d00-4df0-82c2-a7539e8964e4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:48.450593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1084/2181] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:03:48 2022] PUT /identity/v3/projects/80e78c5ecac645ffbc1a9dbc3aa0f82e/users/4e41807dbd8a4922ba199f4bab63a9b5/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:48.469919 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515b788-5d00-4df0-82c2-a7539e8964e4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:48.554305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4059083-06cc-44a2-9d14-be183f9f6ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:48.555725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4059083-06cc-44a2-9d14-be183f9f6ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:48.555860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4059083-06cc-44a2-9d14-be183f9f6ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:48.556163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4059083-06cc-44a2-9d14-be183f9f6ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:48.556869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4059083-06cc-44a2-9d14-be183f9f6ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:48.558826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4059083-06cc-44a2-9d14-be183f9f6ccc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:48.580865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1098/2182] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:03:48.584625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1a83c-ba4e-475f-abef-92388f363d5f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:48.598649 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1a83c-ba4e-475f-abef-92388f363d5f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:48.607900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5987638-ad60-4fee-a556-f0a794e9846d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:48.610444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5987638-ad60-4fee-a556-f0a794e9846d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:48.613339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5987638-ad60-4fee-a556-f0a794e9846d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:48.613524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5987638-ad60-4fee-a556-f0a794e9846d admin admin] PATH_INFO: `/v3/projects/80e78c5ecac645ffbc1a9dbc3aa0f82e/users/4e41807dbd8a4922ba199f4bab63a9b5/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:48.616931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5987638-ad60-4fee-a556-f0a794e9846d admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e78c5ecac645ffbc1a9dbc3aa0f82e, user_id=4e41807dbd8a4922ba199f4bab63a9b5, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:48.621362 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5987638-ad60-4fee-a556-f0a794e9846d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:48.638959 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1085/2183] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:03:48 2022] PUT /identity/v3/projects/80e78c5ecac645ffbc1a9dbc3aa0f82e/users/4e41807dbd8a4922ba199f4bab63a9b5/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:03:54.677029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db3525-eb01-4183-9ad8-e9760ab15bf1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:54.677618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db3525-eb01-4183-9ad8-e9760ab15bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:54.677838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db3525-eb01-4183-9ad8-e9760ab15bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:54.709180 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-05db3525-eb01-4183-9ad8-e9760ab15bf1 None None] MFA Rules not processed for user `4e41807dbd8a4922ba199f4bab63a9b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:54.710417 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-05db3525-eb01-4183-9ad8-e9760ab15bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:54.765779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:54.800153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1099/2184] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:54.923288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5987638-ad60-4fee-a556-f0a794e9846d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:54.925447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c5987638-ad60-4fee-a556-f0a794e9846d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:55.005286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5987638-ad60-4fee-a556-f0a794e9846d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:55.011177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348bab6-56a7-4ec2-8ee0-5b044f16d204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:55.011911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348bab6-56a7-4ec2-8ee0-5b044f16d204 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:55.012315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348bab6-56a7-4ec2-8ee0-5b044f16d204 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:55.012600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348bab6-56a7-4ec2-8ee0-5b044f16d204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:55.033294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:55.075320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348bab6-56a7-4ec2-8ee0-5b044f16d204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:55.076951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348bab6-56a7-4ec2-8ee0-5b044f16d204 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:55.087084 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1086/2185] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:04.796912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=12 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:04:05.081796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=18 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:04:19.883446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6638a-a1b1-4a6b-a188-587d28b7557b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:19.883951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6638a-a1b1-4a6b-a188-587d28b7557b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:19.884136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6638a-a1b1-4a6b-a188-587d28b7557b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:19.903615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-82c6638a-a1b1-4a6b-a188-587d28b7557b None None] MFA Rules not processed for user `6bb5a1f18dff4395bec9c205a9937493`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:19.904866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-82c6638a-a1b1-4a6b-a188-587d28b7557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:19.967337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1100/2186] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:04:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:04:24.808943 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11db86-aa20-4804-aa7f-f7e2c7184a49 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:24.809410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11db86-aa20-4804-aa7f-f7e2c7184a49 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:24.809529 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11db86-aa20-4804-aa7f-f7e2c7184a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:24.827335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0b11db86-aa20-4804-aa7f-f7e2c7184a49 None None] MFA Rules not processed for user `8f8da2142b9e4751b436cfd4920587a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:24.828563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0b11db86-aa20-4804-aa7f-f7e2c7184a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:24.835525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1087/2187] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:04:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:25.477765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d5a9b-2b4e-4905-9fac-95c68ff5e5b5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:25.478282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d5a9b-2b4e-4905-9fac-95c68ff5e5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:25.478378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d5a9b-2b4e-4905-9fac-95c68ff5e5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:25.496516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6d0d5a9b-2b4e-4905-9fac-95c68ff5e5b5 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:25.497813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6d0d5a9b-2b4e-4905-9fac-95c68ff5e5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:25.561189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1101/2188] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:04:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:25.623822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11db86-aa20-4804-aa7f-f7e2c7184a49 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:25.625885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0b11db86-aa20-4804-aa7f-f7e2c7184a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:25.756948 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11db86-aa20-4804-aa7f-f7e2c7184a49 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:25.773970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed20c0a-d272-49ce-83a9-276132a8d4f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:25.775244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed20c0a-d272-49ce-83a9-276132a8d4f7 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:25.775244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed20c0a-d272-49ce-83a9-276132a8d4f7 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:25.775429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed20c0a-d272-49ce-83a9-276132a8d4f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:25.787823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed20c0a-d272-49ce-83a9-276132a8d4f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:25.789945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed20c0a-d272-49ce-83a9-276132a8d4f7 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:25.801617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1088/2189] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:04:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:04:28.813854 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d29a5-d39d-4738-a453-597b9a9e12c8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:28.814479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d29a5-d39d-4738-a453-597b9a9e12c8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:28.814479 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d29a5-d39d-4738-a453-597b9a9e12c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:28.838477 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ab5d29a5-d39d-4738-a453-597b9a9e12c8 None None] MFA Rules not processed for user `df7d511abd8544878e7d0097a1ab0319`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:28.840502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ab5d29a5-d39d-4738-a453-597b9a9e12c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:28.914829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1102/2190] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:35.795819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=35 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:04:38.903947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=75 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:04:48.630913 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde97b84-b539-4159-9a85-11b142761479 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:48.631445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde97b84-b539-4159-9a85-11b142761479 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:48.631565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde97b84-b539-4159-9a85-11b142761479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:48.660923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bde97b84-b539-4159-9a85-11b142761479 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:48.662123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bde97b84-b539-4159-9a85-11b142761479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:48.691578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1089/2191] 10.222.0.65 () {58 vars in 967 bytes} [Thu Jun 23 11:04:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:48.704780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d29a5-d39d-4738-a453-597b9a9e12c8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:48.741130 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d29a5-d39d-4738-a453-597b9a9e12c8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:48.746329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad0edd-6d4d-41ee-8630-1eadb6f0c0e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:48.747013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad0edd-6d4d-41ee-8630-1eadb6f0c0e9 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:48.747297 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad0edd-6d4d-41ee-8630-1eadb6f0c0e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:48.747576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad0edd-6d4d-41ee-8630-1eadb6f0c0e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:48.757365 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad0edd-6d4d-41ee-8630-1eadb6f0c0e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:48.759706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad0edd-6d4d-41ee-8630-1eadb6f0c0e9 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:48.768490 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1103/2192] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:04:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:04:50.827847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde97b84-b539-4159-9a85-11b142761479 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:50.829315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bde97b84-b539-4159-9a85-11b142761479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:50.930572 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde97b84-b539-4159-9a85-11b142761479 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:50.935032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e44668-23dc-42a0-a51c-02c4d5d7af65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:50.935940 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e44668-23dc-42a0-a51c-02c4d5d7af65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:50.936227 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e44668-23dc-42a0-a51c-02c4d5d7af65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:50.936508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e44668-23dc-42a0-a51c-02c4d5d7af65 admin admin] PATH_INFO: `/v3/users/17071129652c4fa09b3b4b20c0dc0725` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:50.937710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e44668-23dc-42a0-a51c-02c4d5d7af65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17071129652c4fa09b3b4b20c0dc0725)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:50.939621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e44668-23dc-42a0-a51c-02c4d5d7af65 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:50.986880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c7e44668-23dc-42a0-a51c-02c4d5d7af65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17071129652c4fa09b3b4b20c0dc0725'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:04:50.987524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c7e44668-23dc-42a0-a51c-02c4d5d7af65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17071129652c4fa09b3b4b20c0dc0725'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:04:50.994828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c7e44668-23dc-42a0-a51c-02c4d5d7af65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17071129652c4fa09b3b4b20c0dc0725'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:04:51.006874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c7e44668-23dc-42a0-a51c-02c4d5d7af65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17071129652c4fa09b3b4b20c0dc0725'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:04:51.014255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c7e44668-23dc-42a0-a51c-02c4d5d7af65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17071129652c4fa09b3b4b20c0dc0725'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:04:51.020206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1090/2193] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:04:50 2022] DELETE /identity/v3/users/17071129652c4fa09b3b4b20c0dc0725 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:04:51.195879 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad0edd-6d4d-41ee-8630-1eadb6f0c0e9 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:51.207442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad0edd-6d4d-41ee-8630-1eadb6f0c0e9 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:51.265824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf10ef3-45cc-4dad-927a-b9769220dc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:51.266767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf10ef3-45cc-4dad-927a-b9769220dc6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:51.267189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf10ef3-45cc-4dad-927a-b9769220dc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:51.267476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf10ef3-45cc-4dad-927a-b9769220dc6e admin admin] PATH_INFO: `/v3/projects/1da9afa4daa046239d16a594150a994c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:51.268704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf10ef3-45cc-4dad-927a-b9769220dc6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da9afa4daa046239d16a594150a994c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:51.270606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf10ef3-45cc-4dad-927a-b9769220dc6e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:51.315800 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9bf10ef3-45cc-4dad-927a-b9769220dc6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da9afa4daa046239d16a594150a994c', 'request_id': 'req-9bf10ef3-45cc-4dad-927a-b9769220dc6e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:04:51.322178 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9bf10ef3-45cc-4dad-927a-b9769220dc6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da9afa4daa046239d16a594150a994c', 'request_id': 'req-9bf10ef3-45cc-4dad-927a-b9769220dc6e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:04:51.328606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9bf10ef3-45cc-4dad-927a-b9769220dc6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da9afa4daa046239d16a594150a994c', 'request_id': 'req-9bf10ef3-45cc-4dad-927a-b9769220dc6e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:04:51.344701 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9bf10ef3-45cc-4dad-927a-b9769220dc6e admin admin] The token cache is being invalidate because project 1da9afa4daa046239d16a594150a994c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:04:51.345203 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9bf10ef3-45cc-4dad-927a-b9769220dc6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:04:51.345834 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9bf10ef3-45cc-4dad-927a-b9769220dc6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:04:51.348084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1104/2194] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:04:51 2022] DELETE /identity/v3/projects/1da9afa4daa046239d16a594150a994c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:04:51.352686 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4e6f1-ce29-4665-a94a-e6ac59698eeb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:51.353076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4e6f1-ce29-4665-a94a-e6ac59698eeb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:51.353459 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4e6f1-ce29-4665-a94a-e6ac59698eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:51.374279 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:04:51.415568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c5a4e6f1-ce29-4665-a94a-e6ac59698eeb None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:51.416865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c5a4e6f1-ce29-4665-a94a-e6ac59698eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:51.479466 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1091/2195] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:04:51.487088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a958e9a-1548-438c-bb7d-2c47fbbee069 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:51.487414 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a958e9a-1548-438c-bb7d-2c47fbbee069 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:51.487649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a958e9a-1548-438c-bb7d-2c47fbbee069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:51.506848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2a958e9a-1548-438c-bb7d-2c47fbbee069 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:51.508098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2a958e9a-1548-438c-bb7d-2c47fbbee069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:51.516728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1105/2196] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:51.520992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4e6f1-ce29-4665-a94a-e6ac59698eeb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:51.533363 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4e6f1-ce29-4665-a94a-e6ac59698eeb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:51.538161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144d11f-dce3-4f6a-95cb-e8f7e8af394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:51.538857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144d11f-dce3-4f6a-95cb-e8f7e8af394f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:51.539161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144d11f-dce3-4f6a-95cb-e8f7e8af394f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:51.539441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144d11f-dce3-4f6a-95cb-e8f7e8af394f admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:51.540354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144d11f-dce3-4f6a-95cb-e8f7e8af394f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:04:51.540665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144d11f-dce3-4f6a-95cb-e8f7e8af394f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:51.542302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144d11f-dce3-4f6a-95cb-e8f7e8af394f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:51.558323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1092/2197] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:04:51.559595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a958e9a-1548-438c-bb7d-2c47fbbee069 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:51.569549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a958e9a-1548-438c-bb7d-2c47fbbee069 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:51.574173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecdf69-2b77-4492-be96-869da31bfc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:51.574828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecdf69-2b77-4492-be96-869da31bfc05 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:51.575109 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecdf69-2b77-4492-be96-869da31bfc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:51.575382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecdf69-2b77-4492-be96-869da31bfc05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:51.576120 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fecdf69-2b77-4492-be96-869da31bfc05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:51.578033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fecdf69-2b77-4492-be96-869da31bfc05 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:51.606049 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1106/2198] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:04:51 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:04:51.607778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144d11f-dce3-4f6a-95cb-e8f7e8af394f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:51.617679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144d11f-dce3-4f6a-95cb-e8f7e8af394f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:51.678447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af4a47-6683-4ffd-9388-6845534fdbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:51.679134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4a47-6683-4ffd-9388-6845534fdbd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:51.679417 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4a47-6683-4ffd-9388-6845534fdbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:51.679721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4a47-6683-4ffd-9388-6845534fdbd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:51.680432 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af4a47-6683-4ffd-9388-6845534fdbd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:51.682420 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af4a47-6683-4ffd-9388-6845534fdbd1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:51.709447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1093/2199] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:04:51 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:04:51.710882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecdf69-2b77-4492-be96-869da31bfc05 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:51.721827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecdf69-2b77-4492-be96-869da31bfc05 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:51.726693 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe97803-f6c3-4278-8141-85b597a54de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:51.727460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe97803-f6c3-4278-8141-85b597a54de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:51.727851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe97803-f6c3-4278-8141-85b597a54de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:51.728239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe97803-f6c3-4278-8141-85b597a54de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:51.729055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe97803-f6c3-4278-8141-85b597a54de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:51.730795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe97803-f6c3-4278-8141-85b597a54de2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:51.742809 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1107/2200] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:04:51.747173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af4a47-6683-4ffd-9388-6845534fdbd1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:51.759187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af4a47-6683-4ffd-9388-6845534fdbd1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:51.772639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be5231-0a16-4fea-9d6e-56381bcd1ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:51.773443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be5231-0a16-4fea-9d6e-56381bcd1ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:51.773839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be5231-0a16-4fea-9d6e-56381bcd1ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:51.774187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be5231-0a16-4fea-9d6e-56381bcd1ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:51.775077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be5231-0a16-4fea-9d6e-56381bcd1ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:51.776756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be5231-0a16-4fea-9d6e-56381bcd1ef6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:51.791173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1094/2201] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:04:51.792807 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe97803-f6c3-4278-8141-85b597a54de2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:51.806230 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe97803-f6c3-4278-8141-85b597a54de2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:51.812540 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244449a-169f-4852-84e3-e6019f0cdb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:51.813735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244449a-169f-4852-84e3-e6019f0cdb5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:51.814266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244449a-169f-4852-84e3-e6019f0cdb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:51.814642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244449a-169f-4852-84e3-e6019f0cdb5c admin admin] PATH_INFO: `/v3/projects/2c49582d70004319ae97ecc62d0bffd0/users/48b1ddbe13a8465994c8fd141f7b355a/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:51.827766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:04:51.832946 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244449a-169f-4852-84e3-e6019f0cdb5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c49582d70004319ae97ecc62d0bffd0, user_id=48b1ddbe13a8465994c8fd141f7b355a, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:51.838655 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244449a-169f-4852-84e3-e6019f0cdb5c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:51.860020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1108/2202] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:04:51 2022] PUT /identity/v3/projects/2c49582d70004319ae97ecc62d0bffd0/users/48b1ddbe13a8465994c8fd141f7b355a/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:04:51.875712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b652cac-3793-4756-8738-978a65a51da9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:51.876559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b652cac-3793-4756-8738-978a65a51da9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:51.876559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b652cac-3793-4756-8738-978a65a51da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:51.916918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8b652cac-3793-4756-8738-978a65a51da9 None None] MFA Rules not processed for user `48b1ddbe13a8465994c8fd141f7b355a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:51.919435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8b652cac-3793-4756-8738-978a65a51da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:51.967262 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:04:51.997891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1095/2203] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:04:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:52.010622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244449a-169f-4852-84e3-e6019f0cdb5c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:52.014140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a244449a-169f-4852-84e3-e6019f0cdb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:52.075947 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244449a-169f-4852-84e3-e6019f0cdb5c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:52.079743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84deb5-1da5-433e-be5e-e09364ee1b16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:52.080395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84deb5-1da5-433e-be5e-e09364ee1b16 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:52.080676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84deb5-1da5-433e-be5e-e09364ee1b16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:52.080950 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84deb5-1da5-433e-be5e-e09364ee1b16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:52.089773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84deb5-1da5-433e-be5e-e09364ee1b16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:52.093347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84deb5-1da5-433e-be5e-e09364ee1b16 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:52.112014 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1109/2204] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:52 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:52.399945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45597ade-f699-4df2-bff7-684d0e88d689 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:52.400240 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45597ade-f699-4df2-bff7-684d0e88d689 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:52.400520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45597ade-f699-4df2-bff7-684d0e88d689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:52.418707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-45597ade-f699-4df2-bff7-684d0e88d689 None None] MFA Rules not processed for user `1b0423a12e39453e9cd6d1e8ab43af60`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:52.419979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-45597ade-f699-4df2-bff7-684d0e88d689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:52.476797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1096/2205] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:04:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:04:57.298105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84deb5-1da5-433e-be5e-e09364ee1b16 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:57.299511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7c84deb5-1da5-433e-be5e-e09364ee1b16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:57.383428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84deb5-1da5-433e-be5e-e09364ee1b16 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:57.387008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0a5bb-1195-426f-bb45-5c87fd814084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:57.387643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0a5bb-1195-426f-bb45-5c87fd814084 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:57.389004 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0a5bb-1195-426f-bb45-5c87fd814084 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:57.389004 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0a5bb-1195-426f-bb45-5c87fd814084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:57.389996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-02a0a5bb-1195-426f-bb45-5c87fd814084 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:57.452499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0a5bb-1195-426f-bb45-5c87fd814084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:57.453951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0a5bb-1195-426f-bb45-5c87fd814084 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:57.463508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1110/2206] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:05:02.470106 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=34 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:05:07.459152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:05:36.870942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45597ade-f699-4df2-bff7-684d0e88d689 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:05:36.872380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-45597ade-f699-4df2-bff7-684d0e88d689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:05:36.965059 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45597ade-f699-4df2-bff7-684d0e88d689 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:05:36.969994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e98b3-0c2c-4e5b-9c1e-e30bc33ffa34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:05:36.970660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e98b3-0c2c-4e5b-9c1e-e30bc33ffa34 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:36.970934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e98b3-0c2c-4e5b-9c1e-e30bc33ffa34 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:36.971230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e98b3-0c2c-4e5b-9c1e-e30bc33ffa34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:36.973654 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-761e98b3-0c2c-4e5b-9c1e-e30bc33ffa34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:05:36.984643 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e98b3-0c2c-4e5b-9c1e-e30bc33ffa34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:05:36.986208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e98b3-0c2c-4e5b-9c1e-e30bc33ffa34 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:05:36.995830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1097/2207] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:05:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:05:46.036415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0a5bb-1195-426f-bb45-5c87fd814084 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:05:46.099053 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0a5bb-1195-426f-bb45-5c87fd814084 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:05:46.104464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:05:46.105366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:46.105628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:46.105904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] PATH_INFO: `/v3/users/48b1ddbe13a8465994c8fd141f7b355a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:46.107131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48b1ddbe13a8465994c8fd141f7b355a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:05:46.109091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:05:46.154040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48b1ddbe13a8465994c8fd141f7b355a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:05:46.154614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48b1ddbe13a8465994c8fd141f7b355a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:05:46.155242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48b1ddbe13a8465994c8fd141f7b355a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:05:46.167253 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48b1ddbe13a8465994c8fd141f7b355a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:05:46.174521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48b1ddbe13a8465994c8fd141f7b355a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:05:46.187833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1111/2208] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:05:46 2022] DELETE /identity/v3/users/48b1ddbe13a8465994c8fd141f7b355a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:05:46.192216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe4c0f-643e-43a4-8dc9-79fa32734be4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:46.192664 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe4c0f-643e-43a4-8dc9-79fa32734be4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:46.192758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe4c0f-643e-43a4-8dc9-79fa32734be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:46.210300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-55fe4c0f-643e-43a4-8dc9-79fa32734be4 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:05:46.211486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-55fe4c0f-643e-43a4-8dc9-79fa32734be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:05:46.269421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1098/2209] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:05:46.278729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:05:46.280127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:05:46.337112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4a11d-03bc-4fdc-acfa-97b553421058 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:05:46.341058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c87b3-7d1c-419a-9350-cddad5a0bd0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:05:46.341688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c87b3-7d1c-419a-9350-cddad5a0bd0b service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:46.341967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c87b3-7d1c-419a-9350-cddad5a0bd0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:46.342237 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c87b3-7d1c-419a-9350-cddad5a0bd0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:46.351550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c87b3-7d1c-419a-9350-cddad5a0bd0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:05:46.352998 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c87b3-7d1c-419a-9350-cddad5a0bd0b service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:05:46.362562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1112/2210] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:05:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:05:46.637038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe4c0f-643e-43a4-8dc9-79fa32734be4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:05:46.638493 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-55fe4c0f-643e-43a4-8dc9-79fa32734be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:05:46.652632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe4c0f-643e-43a4-8dc9-79fa32734be4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:05:46.658043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:05:46.658923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:46.659210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:46.659489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8 admin admin] PATH_INFO: `/v3/projects/2c49582d70004319ae97ecc62d0bffd0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:46.660730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c49582d70004319ae97ecc62d0bffd0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:05:46.663205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:05:46.715259 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c49582d70004319ae97ecc62d0bffd0', 'request_id': 'req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:05:46.728526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c49582d70004319ae97ecc62d0bffd0', 'request_id': 'req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:05:46.735439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c49582d70004319ae97ecc62d0bffd0', 'request_id': 'req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:05:46.736616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:05:46.749493 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8 admin admin] The token cache is being invalidate because project 2c49582d70004319ae97ecc62d0bffd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:05:46.749778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:05:46.750461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4d4e6cc0-5cf8-4ffb-a589-258c9b20c0b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:05:46.752490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1099/2211] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:05:46 2022] DELETE /identity/v3/projects/2c49582d70004319ae97ecc62d0bffd0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:05:46.758304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a48410-c7ae-43c6-bf58-27eee1341eb5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:46.758891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a48410-c7ae-43c6-bf58-27eee1341eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:46.759203 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a48410-c7ae-43c6-bf58-27eee1341eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:46.781256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b2a48410-c7ae-43c6-bf58-27eee1341eb5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:05:46.782455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b2a48410-c7ae-43c6-bf58-27eee1341eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:05:46.800321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:05:46.845028 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1113/2212] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:05:46.853074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134d354-3b59-4a2d-a896-d408d6f8960d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:46.853370 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134d354-3b59-4a2d-a896-d408d6f8960d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:46.853654 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134d354-3b59-4a2d-a896-d408d6f8960d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:46.872136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2134d354-3b59-4a2d-a896-d408d6f8960d None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:05:46.873336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2134d354-3b59-4a2d-a896-d408d6f8960d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:05:46.882249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1100/2213] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:05:46.886518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a48410-c7ae-43c6-bf58-27eee1341eb5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:05:46.896192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a48410-c7ae-43c6-bf58-27eee1341eb5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:05:46.901790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10cf43-5f9a-4fcd-b7d6-188cfffa618a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:05:46.902444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10cf43-5f9a-4fcd-b7d6-188cfffa618a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:46.902727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10cf43-5f9a-4fcd-b7d6-188cfffa618a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:46.903113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10cf43-5f9a-4fcd-b7d6-188cfffa618a admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:46.903878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10cf43-5f9a-4fcd-b7d6-188cfffa618a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:05:46.904185 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10cf43-5f9a-4fcd-b7d6-188cfffa618a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:05:46.905779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10cf43-5f9a-4fcd-b7d6-188cfffa618a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:05:46.917337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1114/2214] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:05:46.921682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134d354-3b59-4a2d-a896-d408d6f8960d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:05:46.930937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134d354-3b59-4a2d-a896-d408d6f8960d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:05:46.935620 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93db8fb-7939-4fde-bbf9-136eefcc5252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:05:46.936297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93db8fb-7939-4fde-bbf9-136eefcc5252 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:46.936582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93db8fb-7939-4fde-bbf9-136eefcc5252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:46.936859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93db8fb-7939-4fde-bbf9-136eefcc5252 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:46.937554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93db8fb-7939-4fde-bbf9-136eefcc5252 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:05:46.939453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93db8fb-7939-4fde-bbf9-136eefcc5252 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:05:46.969001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1101/2215] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:05:46 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:05:46.970106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10cf43-5f9a-4fcd-b7d6-188cfffa618a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:05:46.980307 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10cf43-5f9a-4fcd-b7d6-188cfffa618a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:05:47.038855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f270d5-9f9c-44d7-b0f6-fba3043db837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:05:47.039580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f270d5-9f9c-44d7-b0f6-fba3043db837 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:47.039830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f270d5-9f9c-44d7-b0f6-fba3043db837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:47.040103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f270d5-9f9c-44d7-b0f6-fba3043db837 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:47.040828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f270d5-9f9c-44d7-b0f6-fba3043db837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:05:47.042723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f270d5-9f9c-44d7-b0f6-fba3043db837 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:05:47.064750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1115/2216] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:05:46 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:05:47.068447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93db8fb-7939-4fde-bbf9-136eefcc5252 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:05:47.079582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93db8fb-7939-4fde-bbf9-136eefcc5252 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:05:47.084306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e90440-3183-4d2b-b8c0-fb9a37677b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:05:47.085566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e90440-3183-4d2b-b8c0-fb9a37677b40 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:47.085866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e90440-3183-4d2b-b8c0-fb9a37677b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:47.086547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e90440-3183-4d2b-b8c0-fb9a37677b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:47.087304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e90440-3183-4d2b-b8c0-fb9a37677b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:05:47.088955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e90440-3183-4d2b-b8c0-fb9a37677b40 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:05:47.101327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1102/2217] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:05:47.105662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f270d5-9f9c-44d7-b0f6-fba3043db837 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:05:47.116389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f270d5-9f9c-44d7-b0f6-fba3043db837 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:05:47.121620 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f4a26-875e-4c67-959a-b89063f72914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:05:47.122275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f4a26-875e-4c67-959a-b89063f72914 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:47.122561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f4a26-875e-4c67-959a-b89063f72914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:47.122838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f4a26-875e-4c67-959a-b89063f72914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:47.123599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f4a26-875e-4c67-959a-b89063f72914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:05:47.125464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f4a26-875e-4c67-959a-b89063f72914 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:05:47.136731 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1116/2218] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:05:47.140766 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e90440-3183-4d2b-b8c0-fb9a37677b40 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:05:47.151426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e90440-3183-4d2b-b8c0-fb9a37677b40 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:05:47.157855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e39964-e0fc-4fe6-9364-fe9c88e0b281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:05:47.159410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e39964-e0fc-4fe6-9364-fe9c88e0b281 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:47.159969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e39964-e0fc-4fe6-9364-fe9c88e0b281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:47.160470 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e39964-e0fc-4fe6-9364-fe9c88e0b281 admin admin] PATH_INFO: `/v3/projects/ce4db61b801c40e4a3407c1a4192a07f/users/f995f7cfc91841dd890550b6e24290c0/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:47.178441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e39964-e0fc-4fe6-9364-fe9c88e0b281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4db61b801c40e4a3407c1a4192a07f, user_id=f995f7cfc91841dd890550b6e24290c0, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:05:47.181119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e39964-e0fc-4fe6-9364-fe9c88e0b281 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:05:47.195537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1103/2219] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:05:47 2022] PUT /identity/v3/projects/ce4db61b801c40e4a3407c1a4192a07f/users/f995f7cfc91841dd890550b6e24290c0/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:05:47.205828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1013c-b647-4384-bfbb-b8ba72e75858 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:47.206124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1013c-b647-4384-bfbb-b8ba72e75858 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:47.206403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1013c-b647-4384-bfbb-b8ba72e75858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:47.232830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e1d1013c-b647-4384-bfbb-b8ba72e75858 None None] MFA Rules not processed for user `f995f7cfc91841dd890550b6e24290c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:05:47.233967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e1d1013c-b647-4384-bfbb-b8ba72e75858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:05:47.297070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1117/2220] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:05:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:05:47.309047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e39964-e0fc-4fe6-9364-fe9c88e0b281 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:05:47.310750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c8e39964-e0fc-4fe6-9364-fe9c88e0b281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:05:47.391369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e39964-e0fc-4fe6-9364-fe9c88e0b281 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:05:47.395210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36522e25-40c6-464f-8e86-84ed5e29f0ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:05:47.395882 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36522e25-40c6-464f-8e86-84ed5e29f0ff service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:05:47.396168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36522e25-40c6-464f-8e86-84ed5e29f0ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:05:47.396445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36522e25-40c6-464f-8e86-84ed5e29f0ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:05:47.405301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36522e25-40c6-464f-8e86-84ed5e29f0ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:05:47.406824 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36522e25-40c6-464f-8e86-84ed5e29f0ff service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:05:47.415814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1104/2221] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:05:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:05:57.294200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:05:57.411678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=65,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:06:04.723288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754246dd-049f-4275-96f3-8e42cc3bf5c0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:04.723803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754246dd-049f-4275-96f3-8e42cc3bf5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:04.723918 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754246dd-049f-4275-96f3-8e42cc3bf5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:04.786150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-754246dd-049f-4275-96f3-8e42cc3bf5c0 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:04.787384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-754246dd-049f-4275-96f3-8e42cc3bf5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:04.867833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1118/2222] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:06:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:06:04.913632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36522e25-40c6-464f-8e86-84ed5e29f0ff service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:04.915044 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-36522e25-40c6-464f-8e86-84ed5e29f0ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:05.018526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36522e25-40c6-464f-8e86-84ed5e29f0ff service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:05.024359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd65d0-7eaa-4246-bd45-a4c02f1f2288 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:05.025013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd65d0-7eaa-4246-bd45-a4c02f1f2288 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:05.025300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd65d0-7eaa-4246-bd45-a4c02f1f2288 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:05.025580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd65d0-7eaa-4246-bd45-a4c02f1f2288 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:05.035960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd65d0-7eaa-4246-bd45-a4c02f1f2288 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:05.037566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd65d0-7eaa-4246-bd45-a4c02f1f2288 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:05.046539 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1105/2223] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:06:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:06:09.813265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754246dd-049f-4275-96f3-8e42cc3bf5c0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:09.815324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-754246dd-049f-4275-96f3-8e42cc3bf5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:09.875693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd65d0-7eaa-4246-bd45-a4c02f1f2288 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:09.925874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754246dd-049f-4275-96f3-8e42cc3bf5c0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:09.934925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55dcc1-0af3-47ba-b460-e961e87ae7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:09.936678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55dcc1-0af3-47ba-b460-e961e87ae7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:09.937069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55dcc1-0af3-47ba-b460-e961e87ae7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:09.937448 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55dcc1-0af3-47ba-b460-e961e87ae7e6 admin admin] PATH_INFO: `/v3/users/1b0423a12e39453e9cd6d1e8ab43af60` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:09.939492 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55dcc1-0af3-47ba-b460-e961e87ae7e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b0423a12e39453e9cd6d1e8ab43af60)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:09.941978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55dcc1-0af3-47ba-b460-e961e87ae7e6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:09.951751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd65d0-7eaa-4246-bd45-a4c02f1f2288 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:09.960388 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:09.962673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:09.962673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:09.962895 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] PATH_INFO: `/v3/users/f995f7cfc91841dd890550b6e24290c0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:09.964916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f995f7cfc91841dd890550b6e24290c0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:09.971004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:10.008028 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9e55dcc1-0af3-47ba-b460-e961e87ae7e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b0423a12e39453e9cd6d1e8ab43af60'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.008903 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9e55dcc1-0af3-47ba-b460-e961e87ae7e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b0423a12e39453e9cd6d1e8ab43af60'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.009941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9e55dcc1-0af3-47ba-b460-e961e87ae7e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b0423a12e39453e9cd6d1e8ab43af60'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.020108 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:06:10.029376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9e55dcc1-0af3-47ba-b460-e961e87ae7e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b0423a12e39453e9cd6d1e8ab43af60'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.037619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f995f7cfc91841dd890550b6e24290c0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.038824 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f995f7cfc91841dd890550b6e24290c0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.044139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9e55dcc1-0af3-47ba-b460-e961e87ae7e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b0423a12e39453e9cd6d1e8ab43af60'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.048923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f995f7cfc91841dd890550b6e24290c0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.074364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f995f7cfc91841dd890550b6e24290c0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.074604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1119/2224] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:06:09 2022] DELETE /identity/v3/users/1b0423a12e39453e9cd6d1e8ab43af60 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:06:10.075928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:06:10.085780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f995f7cfc91841dd890550b6e24290c0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.101803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1106/2225] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:06:09 2022] DELETE /identity/v3/users/f995f7cfc91841dd890550b6e24290c0 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:06:10.103290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4f1c3-92e6-4169-8f27-b6bce5530e09 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:10.103596 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4f1c3-92e6-4169-8f27-b6bce5530e09 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:10.103913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4f1c3-92e6-4169-8f27-b6bce5530e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:10.147444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e5d4f1c3-92e6-4169-8f27-b6bce5530e09 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:10.150176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e5d4f1c3-92e6-4169-8f27-b6bce5530e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:10.275566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1120/2226] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:06:10.286927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:10.289279 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:10.356578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4f1c3-92e6-4169-8f27-b6bce5530e09 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:10.358221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e5d4f1c3-92e6-4169-8f27-b6bce5530e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:10.376639 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4f1c3-92e6-4169-8f27-b6bce5530e09 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:10.387078 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fee19-6eb6-4e18-bc44-882b2ff54439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:10.389543 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fee19-6eb6-4e18-bc44-882b2ff54439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:10.390429 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fee19-6eb6-4e18-bc44-882b2ff54439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:10.391071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fee19-6eb6-4e18-bc44-882b2ff54439 admin admin] PATH_INFO: `/v3/projects/64682ed401c74af7aa436a2171819ebe` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:10.391329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c25f54-a283-45be-a59e-f5c97d2aaab3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:10.393220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839fee19-6eb6-4e18-bc44-882b2ff54439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64682ed401c74af7aa436a2171819ebe)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:10.395705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839fee19-6eb6-4e18-bc44-882b2ff54439 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:10.398969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f000443-e7f7-4681-b4a1-40c18f3450f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:10.399453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f000443-e7f7-4681-b4a1-40c18f3450f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:10.399908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f000443-e7f7-4681-b4a1-40c18f3450f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:10.400190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f000443-e7f7-4681-b4a1-40c18f3450f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:10.414986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f000443-e7f7-4681-b4a1-40c18f3450f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:10.416618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f000443-e7f7-4681-b4a1-40c18f3450f3 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:10.429465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1107/2227] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:06:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:06:10.467155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-839fee19-6eb6-4e18-bc44-882b2ff54439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64682ed401c74af7aa436a2171819ebe', 'request_id': 'req-839fee19-6eb6-4e18-bc44-882b2ff54439'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.474138 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-839fee19-6eb6-4e18-bc44-882b2ff54439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64682ed401c74af7aa436a2171819ebe', 'request_id': 'req-839fee19-6eb6-4e18-bc44-882b2ff54439'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.480974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-839fee19-6eb6-4e18-bc44-882b2ff54439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64682ed401c74af7aa436a2171819ebe', 'request_id': 'req-839fee19-6eb6-4e18-bc44-882b2ff54439'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.500740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-839fee19-6eb6-4e18-bc44-882b2ff54439 admin admin] The token cache is being invalidate because project 64682ed401c74af7aa436a2171819ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:06:10.501502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-839fee19-6eb6-4e18-bc44-882b2ff54439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.502456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-839fee19-6eb6-4e18-bc44-882b2ff54439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:10.519214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1121/2228] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:06:10 2022] DELETE /identity/v3/projects/64682ed401c74af7aa436a2171819ebe => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:06:10.795978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f000443-e7f7-4681-b4a1-40c18f3450f3 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:10.798028 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f000443-e7f7-4681-b4a1-40c18f3450f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:10.820284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f000443-e7f7-4681-b4a1-40c18f3450f3 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:10.919702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e593c-db1c-4bfb-83eb-521681c95b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:10.919702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e593c-db1c-4bfb-83eb-521681c95b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:10.919702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e593c-db1c-4bfb-83eb-521681c95b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:10.919702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e593c-db1c-4bfb-83eb-521681c95b0a admin admin] PATH_INFO: `/v3/projects/ce4db61b801c40e4a3407c1a4192a07f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:10.920637 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e593c-db1c-4bfb-83eb-521681c95b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce4db61b801c40e4a3407c1a4192a07f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:10.920637 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e593c-db1c-4bfb-83eb-521681c95b0a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:11.006167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-262e593c-db1c-4bfb-83eb-521681c95b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce4db61b801c40e4a3407c1a4192a07f', 'request_id': 'req-262e593c-db1c-4bfb-83eb-521681c95b0a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:11.014825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-262e593c-db1c-4bfb-83eb-521681c95b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce4db61b801c40e4a3407c1a4192a07f', 'request_id': 'req-262e593c-db1c-4bfb-83eb-521681c95b0a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:11.021622 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-262e593c-db1c-4bfb-83eb-521681c95b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce4db61b801c40e4a3407c1a4192a07f', 'request_id': 'req-262e593c-db1c-4bfb-83eb-521681c95b0a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:11.035296 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-262e593c-db1c-4bfb-83eb-521681c95b0a admin admin] The token cache is being invalidate because project ce4db61b801c40e4a3407c1a4192a07f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:06:11.035597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-262e593c-db1c-4bfb-83eb-521681c95b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:11.036364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-262e593c-db1c-4bfb-83eb-521681c95b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:11.038136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1108/2229] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:06:10 2022] DELETE /identity/v3/projects/ce4db61b801c40e4a3407c1a4192a07f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:06:20.492216 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=62,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:06:21.032686 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:06:21.122867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca86187-7f4d-4f49-bc76-007304407f0b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:21.123160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca86187-7f4d-4f49-bc76-007304407f0b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:21.123442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca86187-7f4d-4f49-bc76-007304407f0b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:21.124865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1122/2230] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 11:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:06:21.142560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078157ef-1470-4e42-b75e-a97116081a97 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:21.144671 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078157ef-1470-4e42-b75e-a97116081a97 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:21.145156 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078157ef-1470-4e42-b75e-a97116081a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:21.232094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-078157ef-1470-4e42-b75e-a97116081a97 None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:21.233585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-078157ef-1470-4e42-b75e-a97116081a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:21.351141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1109/2231] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 11:06:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:06:21.372438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca86187-7f4d-4f49-bc76-007304407f0b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:21.372438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dca86187-7f4d-4f49-bc76-007304407f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:21.469643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca86187-7f4d-4f49-bc76-007304407f0b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:21.473624 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123620ad-63f1-45ea-969b-a4d8ec63810a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af6cc8c56db48beae2550954446c119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:21.474724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123620ad-63f1-45ea-969b-a4d8ec63810a service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:21.475075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123620ad-63f1-45ea-969b-a4d8ec63810a service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:21.475412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123620ad-63f1-45ea-969b-a4d8ec63810a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:21.488840 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123620ad-63f1-45ea-969b-a4d8ec63810a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:21.490433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123620ad-63f1-45ea-969b-a4d8ec63810a service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:21.502678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1123/2232] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:06:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:06:21.588464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d8033-db47-4a3d-884c-6b81d2abaa5b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:21.588763 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d8033-db47-4a3d-884c-6b81d2abaa5b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:21.589071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d8033-db47-4a3d-884c-6b81d2abaa5b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:21.590768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1110/2233] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 11:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:06:21.596166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcf5cb-1075-4ab4-82cb-8f5dcb356b14 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:21.596621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcf5cb-1075-4ab4-82cb-8f5dcb356b14 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:21.596911 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcf5cb-1075-4ab4-82cb-8f5dcb356b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:21.620565 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e6bcf5cb-1075-4ab4-82cb-8f5dcb356b14 None None] MFA Rules not processed for user `e96b136657864104a51900debd1fbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:21.621449 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e6bcf5cb-1075-4ab4-82cb-8f5dcb356b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:21.700371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1124/2234] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 11:06:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:06:21.720919 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d8033-db47-4a3d-884c-6b81d2abaa5b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:21.723093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-832d8033-db47-4a3d-884c-6b81d2abaa5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:21.740070 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d8033-db47-4a3d-884c-6b81d2abaa5b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:21.743731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920d364-b008-4237-8589-c459a232840b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:21.744398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920d364-b008-4237-8589-c459a232840b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:21.744682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920d364-b008-4237-8589-c459a232840b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:21.744960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920d364-b008-4237-8589-c459a232840b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:21.753521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920d364-b008-4237-8589-c459a232840b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:21.755236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920d364-b008-4237-8589-c459a232840b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:21.764673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1111/2235] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:06:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:06:21.853246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae9e00-fb98-4d87-a3fd-4946adf6d048 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:21.853778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae9e00-fb98-4d87-a3fd-4946adf6d048 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:21.854156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae9e00-fb98-4d87-a3fd-4946adf6d048 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:21.855710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1125/2236] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 11:06:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:06:21.861355 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a5d3b-2f10-4a93-9eca-00eea66cf90f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:21.861648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a5d3b-2f10-4a93-9eca-00eea66cf90f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:21.862477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a5d3b-2f10-4a93-9eca-00eea66cf90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:21.881866 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-de3a5d3b-2f10-4a93-9eca-00eea66cf90f None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:21.883097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-de3a5d3b-2f10-4a93-9eca-00eea66cf90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:21.894555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1112/2237] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 11:06:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:06:21.906768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae9e00-fb98-4d87-a3fd-4946adf6d048 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:21.911555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-55ae9e00-fb98-4d87-a3fd-4946adf6d048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:21.934936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae9e00-fb98-4d87-a3fd-4946adf6d048 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:21.940241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02be6d-2c69-4186-b1ef-ac401d415efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:21.941145 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02be6d-2c69-4186-b1ef-ac401d415efd service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:21.941623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02be6d-2c69-4186-b1ef-ac401d415efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:21.942081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02be6d-2c69-4186-b1ef-ac401d415efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:21.954459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02be6d-2c69-4186-b1ef-ac401d415efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:21.956545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02be6d-2c69-4186-b1ef-ac401d415efd service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:21.974200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1126/2238] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:06:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:06:28.625507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c503106-6aa9-416f-90a0-89032985df41 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:28.626071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c503106-6aa9-416f-90a0-89032985df41 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:28.626466 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c503106-6aa9-416f-90a0-89032985df41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:28.653406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5c503106-6aa9-416f-90a0-89032985df41 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:28.655096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5c503106-6aa9-416f-90a0-89032985df41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:28.715796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1113/2239] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:06:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:06:28.728357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02be6d-2c69-4186-b1ef-ac401d415efd service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:28.729827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-db02be6d-2c69-4186-b1ef-ac401d415efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:28.808035 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02be6d-2c69-4186-b1ef-ac401d415efd service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:28.813827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7be28-03c8-4144-a882-cc903d84c47c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:28.814634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7be28-03c8-4144-a882-cc903d84c47c service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:28.814926 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7be28-03c8-4144-a882-cc903d84c47c service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:28.815205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7be28-03c8-4144-a882-cc903d84c47c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:28.828251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7be28-03c8-4144-a882-cc903d84c47c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:28.829865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7be28-03c8-4144-a882-cc903d84c47c service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:28.839711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1127/2240] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:06:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:06:38.708241 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=80 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:06:38.842369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:06:43.654544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9299ab-16a2-4b10-a484-038f4722e122 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:43.655082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9299ab-16a2-4b10-a484-038f4722e122 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:43.655082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9299ab-16a2-4b10-a484-038f4722e122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:43.672468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9b9299ab-16a2-4b10-a484-038f4722e122 None None] MFA Rules not processed for user `6bb5a1f18dff4395bec9c205a9937493`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:43.673696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9b9299ab-16a2-4b10-a484-038f4722e122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:43.731748 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1114/2241] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:06:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:06:46.767349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809f949-f141-4415-8c8e-8b843867bb8a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:46.767882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809f949-f141-4415-8c8e-8b843867bb8a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:46.767975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809f949-f141-4415-8c8e-8b843867bb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:46.785324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8809f949-f141-4415-8c8e-8b843867bb8a None None] MFA Rules not processed for user `df7d511abd8544878e7d0097a1ab0319`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:46.786521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8809f949-f141-4415-8c8e-8b843867bb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:46.840350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1128/2242] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:06:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:06:53.731873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=25 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:06:55.681362 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9299ab-16a2-4b10-a484-038f4722e122 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:55.693007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9299ab-16a2-4b10-a484-038f4722e122 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:55.696754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809f949-f141-4415-8c8e-8b843867bb8a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:55.697238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2b90a-5cb0-498b-ab59-67aa3e0d046b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:55.698604 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2b90a-5cb0-498b-ab59-67aa3e0d046b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:55.698987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2b90a-5cb0-498b-ab59-67aa3e0d046b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:55.699351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2b90a-5cb0-498b-ab59-67aa3e0d046b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:55.701290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9aa2b90a-5cb0-498b-ab59-67aa3e0d046b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:55.708211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809f949-f141-4415-8c8e-8b843867bb8a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:55.713167 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceaef55-90c2-46d3-814f-58cd5a5311cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:55.715151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceaef55-90c2-46d3-814f-58cd5a5311cc service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:55.715734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceaef55-90c2-46d3-814f-58cd5a5311cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:55.716166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceaef55-90c2-46d3-814f-58cd5a5311cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:55.717299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2b90a-5cb0-498b-ab59-67aa3e0d046b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:55.718312 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cceaef55-90c2-46d3-814f-58cd5a5311cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:55.718746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2b90a-5cb0-498b-ab59-67aa3e0d046b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:55.730862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1115/2243] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:06:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:06:55.734766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceaef55-90c2-46d3-814f-58cd5a5311cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:55.736309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceaef55-90c2-46d3-814f-58cd5a5311cc service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:55.749799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1129/2244] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:06:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:06:58.232444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6ef9-6785-451c-b385-340ca54437ae None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:58.232896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6ef9-6785-451c-b385-340ca54437ae None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:58.233216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6ef9-6785-451c-b385-340ca54437ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:58.251046 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-31fb6ef9-6785-451c-b385-340ca54437ae None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:58.253195 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-31fb6ef9-6785-451c-b385-340ca54437ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:58.260987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1116/2245] 10.222.0.65 () {58 vars in 967 bytes} [Thu Jun 23 11:06:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:06:58.273290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceaef55-90c2-46d3-814f-58cd5a5311cc service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:58.285002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceaef55-90c2-46d3-814f-58cd5a5311cc service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:58.290023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccccda-58f2-4dbe-8220-ccec860535cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:58.290685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccccda-58f2-4dbe-8220-ccec860535cb service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:58.290971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccccda-58f2-4dbe-8220-ccec860535cb service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:58.291275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccccda-58f2-4dbe-8220-ccec860535cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:58.299239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccccda-58f2-4dbe-8220-ccec860535cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:58.300907 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccccda-58f2-4dbe-8220-ccec860535cb service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:58.308980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1130/2246] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:06:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:07:08.246039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=9 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:07:08.305583 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=37 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:07:36.518644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6ef9-6785-451c-b385-340ca54437ae None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:07:36.520067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-31fb6ef9-6785-451c-b385-340ca54437ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:36.575238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6ef9-6785-451c-b385-340ca54437ae None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:07:36.578853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9963a2e-68fd-4149-aa46-e9ef94798d40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:07:36.579489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9963a2e-68fd-4149-aa46-e9ef94798d40 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:36.579806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9963a2e-68fd-4149-aa46-e9ef94798d40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:36.580089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9963a2e-68fd-4149-aa46-e9ef94798d40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:36.581833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e9963a2e-68fd-4149-aa46-e9ef94798d40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:36.592436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9963a2e-68fd-4149-aa46-e9ef94798d40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:07:36.593847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9963a2e-68fd-4149-aa46-e9ef94798d40 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:07:36.602681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1117/2247] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:07:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:07:45.092835 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccccda-58f2-4dbe-8220-ccec860535cb service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:07:45.102676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccccda-58f2-4dbe-8220-ccec860535cb service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:07:45.106317 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49457cc7-49fb-4f98-ad00-bdd8fe9153e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:07:45.106955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49457cc7-49fb-4f98-ad00-bdd8fe9153e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:45.107240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49457cc7-49fb-4f98-ad00-bdd8fe9153e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:45.107521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49457cc7-49fb-4f98-ad00-bdd8fe9153e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:45.109897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-49457cc7-49fb-4f98-ad00-bdd8fe9153e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:45.168241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49457cc7-49fb-4f98-ad00-bdd8fe9153e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:07:45.169679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49457cc7-49fb-4f98-ad00-bdd8fe9153e6 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:07:45.179146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1131/2248] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:07:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:07:46.386911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9963a2e-68fd-4149-aa46-e9ef94798d40 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:07:46.388481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e9963a2e-68fd-4149-aa46-e9ef94798d40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:46.403912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9963a2e-68fd-4149-aa46-e9ef94798d40 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:07:46.410430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb887eb-bbcb-46b4-9d1f-7fe5ceaec009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:07:46.411094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb887eb-bbcb-46b4-9d1f-7fe5ceaec009 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:46.411374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb887eb-bbcb-46b4-9d1f-7fe5ceaec009 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:46.411660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb887eb-bbcb-46b4-9d1f-7fe5ceaec009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:46.413496 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7bb887eb-bbcb-46b4-9d1f-7fe5ceaec009 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:46.427035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb887eb-bbcb-46b4-9d1f-7fe5ceaec009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:07:46.428918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb887eb-bbcb-46b4-9d1f-7fe5ceaec009 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:07:46.446713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1118/2249] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:07:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:07:49.173906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49457cc7-49fb-4f98-ad00-bdd8fe9153e6 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:07:49.182575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-49457cc7-49fb-4f98-ad00-bdd8fe9153e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:49.209835 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49457cc7-49fb-4f98-ad00-bdd8fe9153e6 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:07:49.220706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06081e85-c88e-44a4-ba95-e26ff939ac9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:07:49.221381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06081e85-c88e-44a4-ba95-e26ff939ac9f service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:49.221659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06081e85-c88e-44a4-ba95-e26ff939ac9f service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:49.222701 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06081e85-c88e-44a4-ba95-e26ff939ac9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:49.225548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-06081e85-c88e-44a4-ba95-e26ff939ac9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:49.244189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06081e85-c88e-44a4-ba95-e26ff939ac9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:07:49.245976 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06081e85-c88e-44a4-ba95-e26ff939ac9f service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:07:49.256120 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1132/2250] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:07:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:07:49.875675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb887eb-bbcb-46b4-9d1f-7fe5ceaec009 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:07:49.890133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb887eb-bbcb-46b4-9d1f-7fe5ceaec009 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:07:49.903612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb7f56-f1a6-46d1-a406-ffb0d80cff35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:07:49.904668 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb7f56-f1a6-46d1-a406-ffb0d80cff35 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:49.906266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb7f56-f1a6-46d1-a406-ffb0d80cff35 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:49.907256 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb7f56-f1a6-46d1-a406-ffb0d80cff35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:49.914154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4ecb7f56-f1a6-46d1-a406-ffb0d80cff35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:49.935510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb7f56-f1a6-46d1-a406-ffb0d80cff35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:07:49.938922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb7f56-f1a6-46d1-a406-ffb0d80cff35 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:07:49.962172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1119/2251] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:07:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:07:52.945484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9a219-ebd7-4239-8cb9-d888e618006f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:52.947018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9a219-ebd7-4239-8cb9-d888e618006f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:52.947794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9a219-ebd7-4239-8cb9-d888e618006f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:52.983796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c7d9a219-ebd7-4239-8cb9-d888e618006f None None] MFA Rules not processed for user `8f8da2142b9e4751b436cfd4920587a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:07:52.986300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c7d9a219-ebd7-4239-8cb9-d888e618006f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:53.103979 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1133/2252] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:07:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:07:53.493002 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb7f56-f1a6-46d1-a406-ffb0d80cff35 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:07:53.504463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb7f56-f1a6-46d1-a406-ffb0d80cff35 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:07:53.516985 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcdd79e-96b4-4489-91e7-066080a4e7ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:07:53.517895 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdd79e-96b4-4489-91e7-066080a4e7ff service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:53.518268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdd79e-96b4-4489-91e7-066080a4e7ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:53.518571 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdd79e-96b4-4489-91e7-066080a4e7ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:53.520460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0dcdd79e-96b4-4489-91e7-066080a4e7ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:53.531693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcdd79e-96b4-4489-91e7-066080a4e7ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:07:53.533160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcdd79e-96b4-4489-91e7-066080a4e7ff service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:07:53.543299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1120/2253] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:07:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:07:53.933037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d8d86-66b9-4ebd-9996-22a68c64a925 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:53.933248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d8d86-66b9-4ebd-9996-22a68c64a925 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:53.933540 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d8d86-66b9-4ebd-9996-22a68c64a925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:53.953423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-525d8d86-66b9-4ebd-9996-22a68c64a925 None None] MFA Rules not processed for user `303e8aae1bd04228bedb145dec891a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:07:53.955341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-525d8d86-66b9-4ebd-9996-22a68c64a925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:53.967043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1134/2254] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:07:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:07:54.036336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcdd79e-96b4-4489-91e7-066080a4e7ff service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:07:54.037707 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0dcdd79e-96b4-4489-91e7-066080a4e7ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:54.133293 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcdd79e-96b4-4489-91e7-066080a4e7ff service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:07:54.144103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757ca8c-078c-47c6-8042-b696192d346f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:07:54.144956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757ca8c-078c-47c6-8042-b696192d346f service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:54.145375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757ca8c-078c-47c6-8042-b696192d346f service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:54.145753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757ca8c-078c-47c6-8042-b696192d346f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:54.148894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c757ca8c-078c-47c6-8042-b696192d346f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:54.168984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757ca8c-078c-47c6-8042-b696192d346f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:07:54.170706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757ca8c-078c-47c6-8042-b696192d346f service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:07:54.182490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1121/2255] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:07:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:07:54.347856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d8d86-66b9-4ebd-9996-22a68c64a925 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:07:54.362626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d8d86-66b9-4ebd-9996-22a68c64a925 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:07:54.366909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf6a9f-10b8-45e3-8a59-504cfe5787b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:07:54.367993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf6a9f-10b8-45e3-8a59-504cfe5787b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:54.368440 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf6a9f-10b8-45e3-8a59-504cfe5787b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:54.369149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf6a9f-10b8-45e3-8a59-504cfe5787b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:54.371309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8ccf6a9f-10b8-45e3-8a59-504cfe5787b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:54.381675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf6a9f-10b8-45e3-8a59-504cfe5787b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:07:54.383115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf6a9f-10b8-45e3-8a59-504cfe5787b1 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:07:54.392443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1135/2256] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:07:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:07:57.113786 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf520-691d-463d-871a-c5add77fa3c1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:57.114321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf520-691d-463d-871a-c5add77fa3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:57.114321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf520-691d-463d-871a-c5add77fa3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:57.134584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4fcdf520-691d-463d-871a-c5add77fa3c1 None None] MFA Rules not processed for user `94bcc16975124f728459d3c28b97f1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:07:57.136064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4fcdf520-691d-463d-871a-c5add77fa3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:57.202181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:07:57.215841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1122/2257] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:07:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:07:59.476766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf6a9f-10b8-45e3-8a59-504cfe5787b1 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:07:59.486224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf6a9f-10b8-45e3-8a59-504cfe5787b1 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:07:59.487806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf520-691d-463d-871a-c5add77fa3c1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:07:59.491151 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af076fd1-d692-4f8e-8abb-fc2b5daf1ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:07:59.492155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af076fd1-d692-4f8e-8abb-fc2b5daf1ecd service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:59.492502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af076fd1-d692-4f8e-8abb-fc2b5daf1ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:59.492848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af076fd1-d692-4f8e-8abb-fc2b5daf1ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:59.494796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af076fd1-d692-4f8e-8abb-fc2b5daf1ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:07:59.501481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf520-691d-463d-871a-c5add77fa3c1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:07:59.505223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071abcad-ff62-467a-b28b-22480e06b89b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:07:59.506097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071abcad-ff62-467a-b28b-22480e06b89b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:07:59.506383 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071abcad-ff62-467a-b28b-22480e06b89b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:07:59.506664 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071abcad-ff62-467a-b28b-22480e06b89b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:07:59.506985 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af076fd1-d692-4f8e-8abb-fc2b5daf1ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:07:59.508537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af076fd1-d692-4f8e-8abb-fc2b5daf1ecd service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:07:59.518797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1136/2258] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:07:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:07:59.522430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071abcad-ff62-467a-b28b-22480e06b89b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:07:59.524180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071abcad-ff62-467a-b28b-22480e06b89b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:07:59.540616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1123/2259] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:07:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:08:02.001362 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af076fd1-d692-4f8e-8abb-fc2b5daf1ecd service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:02.010195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af076fd1-d692-4f8e-8abb-fc2b5daf1ecd service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:02.013912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4702cf4-0837-411f-bbe2-02e6d4d6fd59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:02.014585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4702cf4-0837-411f-bbe2-02e6d4d6fd59 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:02.014871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4702cf4-0837-411f-bbe2-02e6d4d6fd59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:02.015483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4702cf4-0837-411f-bbe2-02e6d4d6fd59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:02.017454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d4702cf4-0837-411f-bbe2-02e6d4d6fd59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:02.073457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:02.085854 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4702cf4-0837-411f-bbe2-02e6d4d6fd59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:02.087325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4702cf4-0837-411f-bbe2-02e6d4d6fd59 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:02.098153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1137/2260] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:07.022072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071abcad-ff62-467a-b28b-22480e06b89b service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:07.023756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-071abcad-ff62-467a-b28b-22480e06b89b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:07.038669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071abcad-ff62-467a-b28b-22480e06b89b service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:07.043344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd608cfe-7fc4-49ea-af01-1a8c42804eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:07.044491 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd608cfe-7fc4-49ea-af01-1a8c42804eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:07.044752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd608cfe-7fc4-49ea-af01-1a8c42804eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:07.045032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd608cfe-7fc4-49ea-af01-1a8c42804eca admin admin] PATH_INFO: `/v3/users/6bb5a1f18dff4395bec9c205a9937493` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:07.046251 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd608cfe-7fc4-49ea-af01-1a8c42804eca admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb5a1f18dff4395bec9c205a9937493)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:07.048764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd608cfe-7fc4-49ea-af01-1a8c42804eca admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:07.112457 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fd608cfe-7fc4-49ea-af01-1a8c42804eca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb5a1f18dff4395bec9c205a9937493'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:07.113109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fd608cfe-7fc4-49ea-af01-1a8c42804eca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb5a1f18dff4395bec9c205a9937493'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:07.120871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fd608cfe-7fc4-49ea-af01-1a8c42804eca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb5a1f18dff4395bec9c205a9937493'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:07.134433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fd608cfe-7fc4-49ea-af01-1a8c42804eca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb5a1f18dff4395bec9c205a9937493'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:07.142675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fd608cfe-7fc4-49ea-af01-1a8c42804eca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb5a1f18dff4395bec9c205a9937493'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:07.148949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1124/2261] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:08:07 2022] DELETE /identity/v3/users/6bb5a1f18dff4395bec9c205a9937493 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:07.419988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4702cf4-0837-411f-bbe2-02e6d4d6fd59 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:07.445810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4702cf4-0837-411f-bbe2-02e6d4d6fd59 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:07.509390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446dc68-1d89-4868-9d82-738eea0616d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:07.510793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446dc68-1d89-4868-9d82-738eea0616d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:07.511065 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446dc68-1d89-4868-9d82-738eea0616d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:07.511378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446dc68-1d89-4868-9d82-738eea0616d7 admin admin] PATH_INFO: `/v3/projects/d3db3ac9cb13453fa7bc8195bdf88fd1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:07.512619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446dc68-1d89-4868-9d82-738eea0616d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3db3ac9cb13453fa7bc8195bdf88fd1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:07.514678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446dc68-1d89-4868-9d82-738eea0616d7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:07.563149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a446dc68-1d89-4868-9d82-738eea0616d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3db3ac9cb13453fa7bc8195bdf88fd1', 'request_id': 'req-a446dc68-1d89-4868-9d82-738eea0616d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:07.569710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a446dc68-1d89-4868-9d82-738eea0616d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3db3ac9cb13453fa7bc8195bdf88fd1', 'request_id': 'req-a446dc68-1d89-4868-9d82-738eea0616d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:07.575693 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a446dc68-1d89-4868-9d82-738eea0616d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3db3ac9cb13453fa7bc8195bdf88fd1', 'request_id': 'req-a446dc68-1d89-4868-9d82-738eea0616d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:07.589510 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a446dc68-1d89-4868-9d82-738eea0616d7 admin admin] The token cache is being invalidate because project d3db3ac9cb13453fa7bc8195bdf88fd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:08:07.589800 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a446dc68-1d89-4868-9d82-738eea0616d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:07.590605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a446dc68-1d89-4868-9d82-738eea0616d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:07.592426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1138/2262] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:08:07 2022] DELETE /identity/v3/projects/d3db3ac9cb13453fa7bc8195bdf88fd1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:07.597965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbb492-7923-4344-a719-2434d0b57660 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:07.598928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbb492-7923-4344-a719-2434d0b57660 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:07.598928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbb492-7923-4344-a719-2434d0b57660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:07.618194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d8dbb492-7923-4344-a719-2434d0b57660 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:07.619389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d8dbb492-7923-4344-a719-2434d0b57660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:07.690425 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1125/2263] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:08:07.698017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cca06-1d5d-4fc8-b0ee-734a9f832a90 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:07.698415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cca06-1d5d-4fc8-b0ee-734a9f832a90 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:07.698761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cca06-1d5d-4fc8-b0ee-734a9f832a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:07.717598 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c17cca06-1d5d-4fc8-b0ee-734a9f832a90 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:07.719484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c17cca06-1d5d-4fc8-b0ee-734a9f832a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:07.727897 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1139/2264] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:08:07.731747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbb492-7923-4344-a719-2434d0b57660 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:07.742525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbb492-7923-4344-a719-2434d0b57660 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:07.747328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1041e-0423-44f2-883b-5c209b6429da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:07.747971 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1041e-0423-44f2-883b-5c209b6429da admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:07.748285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1041e-0423-44f2-883b-5c209b6429da admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:07.748529 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1041e-0423-44f2-883b-5c209b6429da admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:07.749246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1041e-0423-44f2-883b-5c209b6429da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:08:07.749551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1041e-0423-44f2-883b-5c209b6429da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:07.751767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1041e-0423-44f2-883b-5c209b6429da admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:07.763965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1126/2265] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:08:07.768078 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cca06-1d5d-4fc8-b0ee-734a9f832a90 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:07.778366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cca06-1d5d-4fc8-b0ee-734a9f832a90 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:07.784382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee36ed-a785-4c67-999a-d47ccd0b6b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:07.785047 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee36ed-a785-4c67-999a-d47ccd0b6b9e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:07.785335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee36ed-a785-4c67-999a-d47ccd0b6b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:07.785618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee36ed-a785-4c67-999a-d47ccd0b6b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:07.786333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee36ed-a785-4c67-999a-d47ccd0b6b9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:07.788292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee36ed-a785-4c67-999a-d47ccd0b6b9e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:07.814592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1140/2266] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:08:07 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:08:07.818941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1041e-0423-44f2-883b-5c209b6429da admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:07.829293 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1041e-0423-44f2-883b-5c209b6429da admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:07.878639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:07.905110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c53e3-6b0b-4a48-b36f-521d8cf6bc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:07.906911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c53e3-6b0b-4a48-b36f-521d8cf6bc2c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:07.907201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c53e3-6b0b-4a48-b36f-521d8cf6bc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:07.907476 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c53e3-6b0b-4a48-b36f-521d8cf6bc2c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:07.910279 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c53e3-6b0b-4a48-b36f-521d8cf6bc2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:07.912873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c53e3-6b0b-4a48-b36f-521d8cf6bc2c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:07.966844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1127/2267] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:08:07 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:08:07.967857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee36ed-a785-4c67-999a-d47ccd0b6b9e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:07.983836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee36ed-a785-4c67-999a-d47ccd0b6b9e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:07.989763 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb661d7f-237e-4f73-9063-d5ee94f6fb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:07.990676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb661d7f-237e-4f73-9063-d5ee94f6fb40 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:07.991115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb661d7f-237e-4f73-9063-d5ee94f6fb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:07.991553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb661d7f-237e-4f73-9063-d5ee94f6fb40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:07.992539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb661d7f-237e-4f73-9063-d5ee94f6fb40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:07.994341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb661d7f-237e-4f73-9063-d5ee94f6fb40 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:08.011976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c53e3-6b0b-4a48-b36f-521d8cf6bc2c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:08.018059 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1141/2268] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:08:08.027696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c53e3-6b0b-4a48-b36f-521d8cf6bc2c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:08.034762 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03107aac-fa15-44e9-95fd-56601d348d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:08.036648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03107aac-fa15-44e9-95fd-56601d348d53 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:08.037834 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03107aac-fa15-44e9-95fd-56601d348d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:08.038289 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03107aac-fa15-44e9-95fd-56601d348d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:08.039093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03107aac-fa15-44e9-95fd-56601d348d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:08.041051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03107aac-fa15-44e9-95fd-56601d348d53 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:08.059592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1128/2269] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:08:08.061363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb661d7f-237e-4f73-9063-d5ee94f6fb40 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:08.076521 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb661d7f-237e-4f73-9063-d5ee94f6fb40 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:08.114319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80c96-4bee-4981-b67f-deaab0323f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:08.124539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80c96-4bee-4981-b67f-deaab0323f75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:08.125738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80c96-4bee-4981-b67f-deaab0323f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:08.126284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80c96-4bee-4981-b67f-deaab0323f75 admin admin] PATH_INFO: `/v3/projects/d2cc0729d6c34a4092c5658a96444bf9/users/6969d44a28284bc38d8b17d721986ae3/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:08.163270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f80c96-4bee-4981-b67f-deaab0323f75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2cc0729d6c34a4092c5658a96444bf9, user_id=6969d44a28284bc38d8b17d721986ae3, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:08.168330 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f80c96-4bee-4981-b67f-deaab0323f75 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:08.204875 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1142/2270] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:08:08 2022] PUT /identity/v3/projects/d2cc0729d6c34a4092c5658a96444bf9/users/6969d44a28284bc38d8b17d721986ae3/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:08.215562 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fcccc-0abd-4909-b5a1-36ec499b4248 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:08.215860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fcccc-0abd-4909-b5a1-36ec499b4248 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:08.216139 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fcccc-0abd-4909-b5a1-36ec499b4248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:08.254474 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d42fcccc-0abd-4909-b5a1-36ec499b4248 None None] MFA Rules not processed for user `6969d44a28284bc38d8b17d721986ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:08.256128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d42fcccc-0abd-4909-b5a1-36ec499b4248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:08.354933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1129/2271] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:08.360104 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80c96-4bee-4981-b67f-deaab0323f75 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:08.388857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:08.458954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80c96-4bee-4981-b67f-deaab0323f75 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:08.466265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aee797-aa3d-47bd-bfaf-0beea368c36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:08.467299 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aee797-aa3d-47bd-bfaf-0beea368c36c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:08.469432 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aee797-aa3d-47bd-bfaf-0beea368c36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:08.470053 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aee797-aa3d-47bd-bfaf-0beea368c36c admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:08.471712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aee797-aa3d-47bd-bfaf-0beea368c36c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:08.474798 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aee797-aa3d-47bd-bfaf-0beea368c36c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:08.521723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1143/2272] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:08:08 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:08:08.522922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fcccc-0abd-4909-b5a1-36ec499b4248 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:08.539409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fcccc-0abd-4909-b5a1-36ec499b4248 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:08.616793 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b0f2b-fbc6-4535-9f4b-07f9fea3497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:08.617471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b0f2b-fbc6-4535-9f4b-07f9fea3497a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:08.617777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b0f2b-fbc6-4535-9f4b-07f9fea3497a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:08.618035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b0f2b-fbc6-4535-9f4b-07f9fea3497a admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:08.619541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b0f2b-fbc6-4535-9f4b-07f9fea3497a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:08.622191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b0f2b-fbc6-4535-9f4b-07f9fea3497a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:08.645952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aee797-aa3d-47bd-bfaf-0beea368c36c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:08.647805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-08aee797-aa3d-47bd-bfaf-0beea368c36c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:08.655342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1130/2273] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:08:08 2022] POST /identity/v3/users => generated 295 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:08:08.657044 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b0f2b-fbc6-4535-9f4b-07f9fea3497a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:08.669634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b0f2b-fbc6-4535-9f4b-07f9fea3497a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:08.676140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aecfbe-43e7-494c-aaf9-9cf6005383c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:08.676822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aecfbe-43e7-494c-aaf9-9cf6005383c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:08.677119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aecfbe-43e7-494c-aaf9-9cf6005383c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:08.677399 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aecfbe-43e7-494c-aaf9-9cf6005383c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:08.678261 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aecfbe-43e7-494c-aaf9-9cf6005383c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:08.680090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aecfbe-43e7-494c-aaf9-9cf6005383c7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:08.695541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1131/2274] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:08:08.697870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aecfbe-43e7-494c-aaf9-9cf6005383c7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:08.710379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aecfbe-43e7-494c-aaf9-9cf6005383c7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:08.715290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad286e-33ef-43e7-bc0b-2bf6a6cf5b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:08.716198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad286e-33ef-43e7-bc0b-2bf6a6cf5b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:08.716485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad286e-33ef-43e7-bc0b-2bf6a6cf5b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:08.716768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad286e-33ef-43e7-bc0b-2bf6a6cf5b31 admin admin] PATH_INFO: `/v3/projects/1e580ffbfca34841afbaa5375b919763/users/7f0002926b0b43d495287c5e515d4a76/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:08.735986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad286e-33ef-43e7-bc0b-2bf6a6cf5b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e580ffbfca34841afbaa5375b919763, user_id=7f0002926b0b43d495287c5e515d4a76, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:08.738721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad286e-33ef-43e7-bc0b-2bf6a6cf5b31 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:08.745776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aee797-aa3d-47bd-bfaf-0beea368c36c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:08.756195 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1132/2275] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:08:08 2022] PUT /identity/v3/projects/1e580ffbfca34841afbaa5375b919763/users/7f0002926b0b43d495287c5e515d4a76/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:08.760369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad286e-33ef-43e7-bc0b-2bf6a6cf5b31 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:08.773693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:08.831784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3202d2-22c1-4bb8-b40e-ab40cf014a49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:08.832442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3202d2-22c1-4bb8-b40e-ab40cf014a49 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:08.832724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3202d2-22c1-4bb8-b40e-ab40cf014a49 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:08.832997 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3202d2-22c1-4bb8-b40e-ab40cf014a49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:08.834962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ae3202d2-22c1-4bb8-b40e-ab40cf014a49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:08.840540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad286e-33ef-43e7-bc0b-2bf6a6cf5b31 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:08.847557 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7104dc-8711-43c0-913e-af0a6518282d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:08.848265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7104dc-8711-43c0-913e-af0a6518282d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:08.848396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3202d2-22c1-4bb8-b40e-ab40cf014a49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:08.848548 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7104dc-8711-43c0-913e-af0a6518282d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:08.848830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7104dc-8711-43c0-913e-af0a6518282d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:08.849561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7104dc-8711-43c0-913e-af0a6518282d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:08.849944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3202d2-22c1-4bb8-b40e-ab40cf014a49 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:08.851151 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7104dc-8711-43c0-913e-af0a6518282d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:08.863838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1144/2276] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:08:08.871931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1133/2277] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:08:08.877127 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3202d2-22c1-4bb8-b40e-ab40cf014a49 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:08.889229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3202d2-22c1-4bb8-b40e-ab40cf014a49 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:08.896384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5975d-c1a3-4e84-bb30-572d07453c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:08.897350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5975d-c1a3-4e84-bb30-572d07453c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:08.897633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5975d-c1a3-4e84-bb30-572d07453c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:08.898005 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5975d-c1a3-4e84-bb30-572d07453c13 admin admin] PATH_INFO: `/v3/projects/1e580ffbfca34841afbaa5375b919763/users/7f0002926b0b43d495287c5e515d4a76/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:08.899974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5975d-c1a3-4e84-bb30-572d07453c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e580ffbfca34841afbaa5375b919763, user_id=7f0002926b0b43d495287c5e515d4a76, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:08.902634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5975d-c1a3-4e84-bb30-572d07453c13 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:08.914155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1145/2278] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:08:08 2022] PUT /identity/v3/projects/1e580ffbfca34841afbaa5375b919763/users/7f0002926b0b43d495287c5e515d4a76/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:08.925921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6544d0d-a9e0-45f2-ad16-90e752e488ab None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:08.926213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6544d0d-a9e0-45f2-ad16-90e752e488ab None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:08.926498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6544d0d-a9e0-45f2-ad16-90e752e488ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:08.952711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f6544d0d-a9e0-45f2-ad16-90e752e488ab None None] MFA Rules not processed for user `7f0002926b0b43d495287c5e515d4a76`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:08.953960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f6544d0d-a9e0-45f2-ad16-90e752e488ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:09.026597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1134/2279] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:09.037130 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5975d-c1a3-4e84-bb30-572d07453c13 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:09.038888 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8ec5975d-c1a3-4e84-bb30-572d07453c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:09.053399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:09.108273 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5975d-c1a3-4e84-bb30-572d07453c13 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:09.112090 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48dcb7-55aa-4baf-ad68-37654741d49b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:09.112744 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48dcb7-55aa-4baf-ad68-37654741d49b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:09.113027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48dcb7-55aa-4baf-ad68-37654741d49b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:09.113307 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48dcb7-55aa-4baf-ad68-37654741d49b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:09.172426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48dcb7-55aa-4baf-ad68-37654741d49b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:09.173846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48dcb7-55aa-4baf-ad68-37654741d49b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:09.183556 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1146/2280] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:11.570467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950199c7-00a9-4a42-8b1b-8e20610e9316 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:11.571009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950199c7-00a9-4a42-8b1b-8e20610e9316 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:11.571009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950199c7-00a9-4a42-8b1b-8e20610e9316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:11.589318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-950199c7-00a9-4a42-8b1b-8e20610e9316 None None] MFA Rules not processed for user `df7d511abd8544878e7d0097a1ab0319`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:11.590521 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-950199c7-00a9-4a42-8b1b-8e20610e9316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:11.646872 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1135/2281] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:13.936266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48dcb7-55aa-4baf-ad68-37654741d49b service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:13.946153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48dcb7-55aa-4baf-ad68-37654741d49b service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:13.950172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca3a28-4d8b-4e42-ae93-663c8be4e195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:13.950926 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca3a28-4d8b-4e42-ae93-663c8be4e195 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:13.951314 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca3a28-4d8b-4e42-ae93-663c8be4e195 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:13.951728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca3a28-4d8b-4e42-ae93-663c8be4e195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:13.970444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca3a28-4d8b-4e42-ae93-663c8be4e195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:13.972523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca3a28-4d8b-4e42-ae93-663c8be4e195 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:14.008287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1147/2282] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:21.639623 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:23.979488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=48 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:39.106189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950199c7-00a9-4a42-8b1b-8e20610e9316 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:39.167354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950199c7-00a9-4a42-8b1b-8e20610e9316 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:39.171848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:39.172709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:39.172989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:39.173263 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] PATH_INFO: `/v3/users/6969d44a28284bc38d8b17d721986ae3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:39.174434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6969d44a28284bc38d8b17d721986ae3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:39.176414 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:39.220734 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6969d44a28284bc38d8b17d721986ae3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.221509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6969d44a28284bc38d8b17d721986ae3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.229782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6969d44a28284bc38d8b17d721986ae3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.241911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6969d44a28284bc38d8b17d721986ae3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.248638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6969d44a28284bc38d8b17d721986ae3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.254820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1136/2283] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:08:39 2022] DELETE /identity/v3/users/6969d44a28284bc38d8b17d721986ae3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:39.259361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c5359-697e-4a22-8556-b7f9db1ed941 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:39.259920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c5359-697e-4a22-8556-b7f9db1ed941 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:39.259920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c5359-697e-4a22-8556-b7f9db1ed941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:39.277331 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5f9c5359-697e-4a22-8556-b7f9db1ed941 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:39.278547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5f9c5359-697e-4a22-8556-b7f9db1ed941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:39.337201 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1148/2284] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:39.347201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:39.348628 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:39.405722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8deaa-8496-432b-9a1d-26f85bb6b9ae admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:39.409920 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d25db-1764-48b9-b67a-7e7822603499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:39.410554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d25db-1764-48b9-b67a-7e7822603499 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:39.410831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d25db-1764-48b9-b67a-7e7822603499 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:39.411100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d25db-1764-48b9-b67a-7e7822603499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:39.419910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d25db-1764-48b9-b67a-7e7822603499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:39.421315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d25db-1764-48b9-b67a-7e7822603499 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:39.430456 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1137/2285] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:39.699665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c5359-697e-4a22-8556-b7f9db1ed941 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:39.701265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5f9c5359-697e-4a22-8556-b7f9db1ed941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:39.714369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c5359-697e-4a22-8556-b7f9db1ed941 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:39.719016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:39.719983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:39.720353 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:39.720648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] PATH_INFO: `/v3/projects/d2cc0729d6c34a4092c5658a96444bf9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:39.721877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2cc0729d6c34a4092c5658a96444bf9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:39.723815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:39.770918 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2cc0729d6c34a4092c5658a96444bf9', 'request_id': 'req-6c5b26f0-b371-488f-8e6b-6c5ead044007'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.776984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2cc0729d6c34a4092c5658a96444bf9', 'request_id': 'req-6c5b26f0-b371-488f-8e6b-6c5ead044007'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.782603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2cc0729d6c34a4092c5658a96444bf9', 'request_id': 'req-6c5b26f0-b371-488f-8e6b-6c5ead044007'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.796734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] The token cache is being invalidate because project d2cc0729d6c34a4092c5658a96444bf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:08:39.797028 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.797676 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.799545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1149/2286] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:08:39 2022] DELETE /identity/v3/projects/d2cc0729d6c34a4092c5658a96444bf9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:39.803260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d25db-1764-48b9-b67a-7e7822603499 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:39.812945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d25db-1764-48b9-b67a-7e7822603499 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:39.831501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:39.871172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7e093-66f6-4fc2-acf9-84f7d18bd524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:39.871996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7e093-66f6-4fc2-acf9-84f7d18bd524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:39.872278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7e093-66f6-4fc2-acf9-84f7d18bd524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:39.872552 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7e093-66f6-4fc2-acf9-84f7d18bd524 admin admin] PATH_INFO: `/v3/users/7f0002926b0b43d495287c5e515d4a76` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:39.873733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7e093-66f6-4fc2-acf9-84f7d18bd524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f0002926b0b43d495287c5e515d4a76)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:39.875648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7e093-66f6-4fc2-acf9-84f7d18bd524 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:39.921723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-eba7e093-66f6-4fc2-acf9-84f7d18bd524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f0002926b0b43d495287c5e515d4a76'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.922366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-eba7e093-66f6-4fc2-acf9-84f7d18bd524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f0002926b0b43d495287c5e515d4a76'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.930209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-eba7e093-66f6-4fc2-acf9-84f7d18bd524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f0002926b0b43d495287c5e515d4a76'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.942856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-eba7e093-66f6-4fc2-acf9-84f7d18bd524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f0002926b0b43d495287c5e515d4a76'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.949563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-eba7e093-66f6-4fc2-acf9-84f7d18bd524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f0002926b0b43d495287c5e515d4a76'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:39.955337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1138/2287] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:08:39 2022] DELETE /identity/v3/users/7f0002926b0b43d495287c5e515d4a76 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:40.222551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:40.224133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:40.239624 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b26f0-b371-488f-8e6b-6c5ead044007 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:40.297799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2950fe-9e0f-4785-ac73-82170f199d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:40.298655 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2950fe-9e0f-4785-ac73-82170f199d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:40.298961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2950fe-9e0f-4785-ac73-82170f199d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:40.299239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2950fe-9e0f-4785-ac73-82170f199d5d admin admin] PATH_INFO: `/v3/projects/1e580ffbfca34841afbaa5375b919763` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:40.300993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2950fe-9e0f-4785-ac73-82170f199d5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e580ffbfca34841afbaa5375b919763)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:40.302886 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2950fe-9e0f-4785-ac73-82170f199d5d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:40.331348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:40.349136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7d2950fe-9e0f-4785-ac73-82170f199d5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e580ffbfca34841afbaa5375b919763', 'request_id': 'req-7d2950fe-9e0f-4785-ac73-82170f199d5d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:40.355082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7d2950fe-9e0f-4785-ac73-82170f199d5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e580ffbfca34841afbaa5375b919763', 'request_id': 'req-7d2950fe-9e0f-4785-ac73-82170f199d5d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:40.360578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7d2950fe-9e0f-4785-ac73-82170f199d5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e580ffbfca34841afbaa5375b919763', 'request_id': 'req-7d2950fe-9e0f-4785-ac73-82170f199d5d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:40.373239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7d2950fe-9e0f-4785-ac73-82170f199d5d admin admin] The token cache is being invalidate because project 1e580ffbfca34841afbaa5375b919763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:08:40.373528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7d2950fe-9e0f-4785-ac73-82170f199d5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:40.374176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7d2950fe-9e0f-4785-ac73-82170f199d5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:40.375733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1150/2288] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:08:40 2022] DELETE /identity/v3/projects/1e580ffbfca34841afbaa5375b919763 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:40.381665 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0955079-3fdd-45cc-8ba4-72990563ba75 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:40.382148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0955079-3fdd-45cc-8ba4-72990563ba75 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:40.382258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0955079-3fdd-45cc-8ba4-72990563ba75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:40.400750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f0955079-3fdd-45cc-8ba4-72990563ba75 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:40.401889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f0955079-3fdd-45cc-8ba4-72990563ba75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:40.459731 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1139/2289] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:08:40.467468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ca050-07d5-4409-85ed-649cc298f088 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:40.467810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ca050-07d5-4409-85ed-649cc298f088 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:40.468096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ca050-07d5-4409-85ed-649cc298f088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:40.486647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-512ca050-07d5-4409-85ed-649cc298f088 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:40.488512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-512ca050-07d5-4409-85ed-649cc298f088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:40.496415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1151/2290] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:40.500381 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0955079-3fdd-45cc-8ba4-72990563ba75 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:40.509945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0955079-3fdd-45cc-8ba4-72990563ba75 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:40.514698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebfc07-03a9-4ce0-b48e-eb37fe172870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:40.515335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebfc07-03a9-4ce0-b48e-eb37fe172870 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:40.515615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebfc07-03a9-4ce0-b48e-eb37fe172870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:40.515928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebfc07-03a9-4ce0-b48e-eb37fe172870 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:40.516674 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebfc07-03a9-4ce0-b48e-eb37fe172870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:08:40.516977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebfc07-03a9-4ce0-b48e-eb37fe172870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:40.518576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebfc07-03a9-4ce0-b48e-eb37fe172870 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:40.529723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1140/2291] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:08:40.533964 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ca050-07d5-4409-85ed-649cc298f088 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:40.543691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ca050-07d5-4409-85ed-649cc298f088 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:40.548318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd6782-dd47-4dc0-8360-33082701aef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:40.548959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd6782-dd47-4dc0-8360-33082701aef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:40.549246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd6782-dd47-4dc0-8360-33082701aef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:40.549526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd6782-dd47-4dc0-8360-33082701aef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:40.550851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd6782-dd47-4dc0-8360-33082701aef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:40.552791 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd6782-dd47-4dc0-8360-33082701aef9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:40.586413 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1152/2292] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:08:40 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:08:40.590434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebfc07-03a9-4ce0-b48e-eb37fe172870 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:40.601087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebfc07-03a9-4ce0-b48e-eb37fe172870 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:40.626926 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:40.661310 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da852fb0-d547-4b0e-a014-17c059625835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:40.661895 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da852fb0-d547-4b0e-a014-17c059625835 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:40.662178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da852fb0-d547-4b0e-a014-17c059625835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:40.662453 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da852fb0-d547-4b0e-a014-17c059625835 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:40.663134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da852fb0-d547-4b0e-a014-17c059625835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:40.665078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da852fb0-d547-4b0e-a014-17c059625835 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:40.685067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1141/2293] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:08:40 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:08:40.689175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd6782-dd47-4dc0-8360-33082701aef9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:40.699227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd6782-dd47-4dc0-8360-33082701aef9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:40.703874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6fc5d-359b-4fe2-acdf-03f2bdc9f0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:40.704531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6fc5d-359b-4fe2-acdf-03f2bdc9f0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:40.704815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6fc5d-359b-4fe2-acdf-03f2bdc9f0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:40.705093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6fc5d-359b-4fe2-acdf-03f2bdc9f0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:40.705824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6fc5d-359b-4fe2-acdf-03f2bdc9f0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:40.707962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6fc5d-359b-4fe2-acdf-03f2bdc9f0b9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:40.717811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1153/2294] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:08:40.722174 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da852fb0-d547-4b0e-a014-17c059625835 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:40.731918 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da852fb0-d547-4b0e-a014-17c059625835 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:40.736485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefa26d-656f-422c-b80d-48fbefab8ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:40.737134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefa26d-656f-422c-b80d-48fbefab8ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:40.737414 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefa26d-656f-422c-b80d-48fbefab8ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:40.737690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefa26d-656f-422c-b80d-48fbefab8ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:40.738409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aefa26d-656f-422c-b80d-48fbefab8ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:40.740034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aefa26d-656f-422c-b80d-48fbefab8ee9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:40.749651 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1142/2295] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:08:40.753514 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6fc5d-359b-4fe2-acdf-03f2bdc9f0b9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:40.763478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6fc5d-359b-4fe2-acdf-03f2bdc9f0b9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:40.768066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a08b9-adbf-4469-91e1-ff1ae0f41b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:40.769424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a08b9-adbf-4469-91e1-ff1ae0f41b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:40.769702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a08b9-adbf-4469-91e1-ff1ae0f41b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:40.769977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a08b9-adbf-4469-91e1-ff1ae0f41b2c admin admin] PATH_INFO: `/v3/projects/d1a722f5cc254ed3809a55c15315afd0/users/716aff96a04b46adb17cb7d254186377/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:40.785263 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a08b9-adbf-4469-91e1-ff1ae0f41b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a722f5cc254ed3809a55c15315afd0, user_id=716aff96a04b46adb17cb7d254186377, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:40.787949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a08b9-adbf-4469-91e1-ff1ae0f41b2c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:40.800575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1154/2296] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:08:40 2022] PUT /identity/v3/projects/d1a722f5cc254ed3809a55c15315afd0/users/716aff96a04b46adb17cb7d254186377/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:08:40.811863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c11428-f07d-4bca-a98d-e5964f9d565a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:40.812154 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c11428-f07d-4bca-a98d-e5964f9d565a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:40.812433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c11428-f07d-4bca-a98d-e5964f9d565a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:40.838214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-07c11428-f07d-4bca-a98d-e5964f9d565a None None] MFA Rules not processed for user `716aff96a04b46adb17cb7d254186377`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:40.839418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-07c11428-f07d-4bca-a98d-e5964f9d565a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:40.901789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1143/2297] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:40.906187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a08b9-adbf-4469-91e1-ff1ae0f41b2c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:40.966451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a08b9-adbf-4469-91e1-ff1ae0f41b2c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:40.972606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d43ac-9f02-4602-af27-791bb4ad0e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:40.973267 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d43ac-9f02-4602-af27-791bb4ad0e84 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:40.973553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d43ac-9f02-4602-af27-791bb4ad0e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:40.973833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d43ac-9f02-4602-af27-791bb4ad0e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:40.974530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d43ac-9f02-4602-af27-791bb4ad0e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:40.976474 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d43ac-9f02-4602-af27-791bb4ad0e84 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:41.001372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1155/2298] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:08:40 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:08:41.006028 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c11428-f07d-4bca-a98d-e5964f9d565a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:41.016860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c11428-f07d-4bca-a98d-e5964f9d565a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:41.071992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26daf0-d320-407b-9c5a-e96f077948bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:41.072662 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26daf0-d320-407b-9c5a-e96f077948bd admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:41.072944 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26daf0-d320-407b-9c5a-e96f077948bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:41.073223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26daf0-d320-407b-9c5a-e96f077948bd admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:41.073923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26daf0-d320-407b-9c5a-e96f077948bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:41.075860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26daf0-d320-407b-9c5a-e96f077948bd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:41.097441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1144/2299] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:08:41 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:08:41.101494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d43ac-9f02-4602-af27-791bb4ad0e84 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:41.112110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d43ac-9f02-4602-af27-791bb4ad0e84 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:41.116732 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57323ba-671d-45cf-b197-c2193963fe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:41.117385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57323ba-671d-45cf-b197-c2193963fe68 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:41.117670 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57323ba-671d-45cf-b197-c2193963fe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:41.117950 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57323ba-671d-45cf-b197-c2193963fe68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:41.118681 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57323ba-671d-45cf-b197-c2193963fe68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:41.120319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57323ba-671d-45cf-b197-c2193963fe68 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:41.131163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1156/2300] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:08:41.135068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26daf0-d320-407b-9c5a-e96f077948bd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:41.145789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26daf0-d320-407b-9c5a-e96f077948bd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:41.150503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9840f8b-4deb-4afe-880e-3742102cbb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:41.151436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9840f8b-4deb-4afe-880e-3742102cbb7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:41.151745 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9840f8b-4deb-4afe-880e-3742102cbb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:41.152017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9840f8b-4deb-4afe-880e-3742102cbb7b admin admin] PATH_INFO: `/v3/projects/97aa82e640744950bd821680ba7eba4f/users/f29a27cffb8847f4a1111c308caa1b02/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:41.166503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9840f8b-4deb-4afe-880e-3742102cbb7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=97aa82e640744950bd821680ba7eba4f, user_id=f29a27cffb8847f4a1111c308caa1b02, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:41.169210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9840f8b-4deb-4afe-880e-3742102cbb7b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:41.181321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1145/2301] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:08:41 2022] PUT /identity/v3/projects/97aa82e640744950bd821680ba7eba4f/users/f29a27cffb8847f4a1111c308caa1b02/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:08:41.185260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57323ba-671d-45cf-b197-c2193963fe68 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:41.195893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57323ba-671d-45cf-b197-c2193963fe68 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:41.207894 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:41.253325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf060005-0e0c-453d-b079-3994c8806f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:41.253951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf060005-0e0c-453d-b079-3994c8806f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:41.254240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf060005-0e0c-453d-b079-3994c8806f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:41.254519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf060005-0e0c-453d-b079-3994c8806f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:41.255240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf060005-0e0c-453d-b079-3994c8806f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:41.256943 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf060005-0e0c-453d-b079-3994c8806f53 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:41.266820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1157/2302] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:08:41.271501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9840f8b-4deb-4afe-880e-3742102cbb7b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:41.276946 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:41.283434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9840f8b-4deb-4afe-880e-3742102cbb7b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:41.288309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc5356-e454-491a-9c23-a9782f811aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:41.289186 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc5356-e454-491a-9c23-a9782f811aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:41.289463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc5356-e454-491a-9c23-a9782f811aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:41.289733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc5356-e454-491a-9c23-a9782f811aaf admin admin] PATH_INFO: `/v3/projects/97aa82e640744950bd821680ba7eba4f/users/f29a27cffb8847f4a1111c308caa1b02/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:41.291690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc5356-e454-491a-9c23-a9782f811aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=97aa82e640744950bd821680ba7eba4f, user_id=f29a27cffb8847f4a1111c308caa1b02, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:41.294317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc5356-e454-491a-9c23-a9782f811aaf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:41.304391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1146/2303] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:08:41 2022] PUT /identity/v3/projects/97aa82e640744950bd821680ba7eba4f/users/f29a27cffb8847f4a1111c308caa1b02/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:41.314329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee38bb6-555c-4989-8b36-40737a4d41b3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:41.314834 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee38bb6-555c-4989-8b36-40737a4d41b3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:41.314969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee38bb6-555c-4989-8b36-40737a4d41b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:41.341380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dee38bb6-555c-4989-8b36-40737a4d41b3 None None] MFA Rules not processed for user `f29a27cffb8847f4a1111c308caa1b02`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:41.342578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dee38bb6-555c-4989-8b36-40737a4d41b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:41.408084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1158/2304] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:41.418349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc5356-e454-491a-9c23-a9782f811aaf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:41.420076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7efc5356-e454-491a-9c23-a9782f811aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:41.478182 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc5356-e454-491a-9c23-a9782f811aaf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:41.481971 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40d77d-2555-4c1d-b7e5-671bd99c9758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:41.482612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40d77d-2555-4c1d-b7e5-671bd99c9758 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:41.482897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40d77d-2555-4c1d-b7e5-671bd99c9758 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:41.483177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40d77d-2555-4c1d-b7e5-671bd99c9758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:41.538212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40d77d-2555-4c1d-b7e5-671bd99c9758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:41.539642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40d77d-2555-4c1d-b7e5-671bd99c9758 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:41.548445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1147/2305] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:45.668799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee38bb6-555c-4989-8b36-40737a4d41b3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:45.670221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dee38bb6-555c-4989-8b36-40737a4d41b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:45.682007 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee38bb6-555c-4989-8b36-40737a4d41b3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:45.685589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86887f58-8f97-436b-a7fa-baf9fb477cf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:45.686228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86887f58-8f97-436b-a7fa-baf9fb477cf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:45.686514 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86887f58-8f97-436b-a7fa-baf9fb477cf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:45.686796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86887f58-8f97-436b-a7fa-baf9fb477cf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:45.696383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86887f58-8f97-436b-a7fa-baf9fb477cf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:45.697806 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86887f58-8f97-436b-a7fa-baf9fb477cf5 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:45.707825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1159/2306] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:50.526771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40d77d-2555-4c1d-b7e5-671bd99c9758 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:50.587058 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40d77d-2555-4c1d-b7e5-671bd99c9758 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:50.591767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:50.592638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:50.592921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:50.593198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] PATH_INFO: `/v3/users/716aff96a04b46adb17cb7d254186377` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:50.594386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=716aff96a04b46adb17cb7d254186377)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:50.596365 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:50.642811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:50.644398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '716aff96a04b46adb17cb7d254186377'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:50.645124 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '716aff96a04b46adb17cb7d254186377'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:50.653298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '716aff96a04b46adb17cb7d254186377'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:50.665800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '716aff96a04b46adb17cb7d254186377'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:50.672481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '716aff96a04b46adb17cb7d254186377'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:50.678844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1148/2307] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:08:50 2022] DELETE /identity/v3/users/716aff96a04b46adb17cb7d254186377 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:50.683292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64302e-5189-4272-bb92-0d6c42c1e37e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:50.683771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64302e-5189-4272-bb92-0d6c42c1e37e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:50.683874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64302e-5189-4272-bb92-0d6c42c1e37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:50.701866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-af64302e-5189-4272-bb92-0d6c42c1e37e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:50.703066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af64302e-5189-4272-bb92-0d6c42c1e37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:50.759545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1160/2308] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:50.769255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:50.770653 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:50.826390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3741daa-9df5-4e85-911f-3fadf5a8af5e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:50.830010 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f6d4b-7e35-40d1-a1e7-8909d0709a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:50.830646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f6d4b-7e35-40d1-a1e7-8909d0709a4b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:50.830928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f6d4b-7e35-40d1-a1e7-8909d0709a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:50.831205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f6d4b-7e35-40d1-a1e7-8909d0709a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:50.840201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f6d4b-7e35-40d1-a1e7-8909d0709a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:50.841610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f6d4b-7e35-40d1-a1e7-8909d0709a4b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:50.851474 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1149/2309] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:51.111752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64302e-5189-4272-bb92-0d6c42c1e37e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:51.113331 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af64302e-5189-4272-bb92-0d6c42c1e37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:51.125536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64302e-5189-4272-bb92-0d6c42c1e37e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:51.130598 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:51.131530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:51.131864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:51.132148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] PATH_INFO: `/v3/projects/d1a722f5cc254ed3809a55c15315afd0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:51.133359 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a722f5cc254ed3809a55c15315afd0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:51.135281 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:51.172891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:51.197514 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a722f5cc254ed3809a55c15315afd0', 'request_id': 'req-2bda5696-16fc-4959-985e-730ae180eff8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.203627 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a722f5cc254ed3809a55c15315afd0', 'request_id': 'req-2bda5696-16fc-4959-985e-730ae180eff8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.209743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a722f5cc254ed3809a55c15315afd0', 'request_id': 'req-2bda5696-16fc-4959-985e-730ae180eff8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.221733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] The token cache is being invalidate because project d1a722f5cc254ed3809a55c15315afd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:08:51.222019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.222643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.224843 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1161/2310] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:08:51 2022] DELETE /identity/v3/projects/d1a722f5cc254ed3809a55c15315afd0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:51.228823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f6d4b-7e35-40d1-a1e7-8909d0709a4b service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:51.238284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f6d4b-7e35-40d1-a1e7-8909d0709a4b service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:51.293289 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fdfa9-fb18-4f9f-8491-031a4a52ac97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:51.294245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fdfa9-fb18-4f9f-8491-031a4a52ac97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:51.294526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fdfa9-fb18-4f9f-8491-031a4a52ac97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:51.294805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fdfa9-fb18-4f9f-8491-031a4a52ac97 admin admin] PATH_INFO: `/v3/users/f29a27cffb8847f4a1111c308caa1b02` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:51.296017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fdfa9-fb18-4f9f-8491-031a4a52ac97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f29a27cffb8847f4a1111c308caa1b02)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:51.297928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fdfa9-fb18-4f9f-8491-031a4a52ac97 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:51.342250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a25fdfa9-fb18-4f9f-8491-031a4a52ac97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f29a27cffb8847f4a1111c308caa1b02'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.342886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a25fdfa9-fb18-4f9f-8491-031a4a52ac97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f29a27cffb8847f4a1111c308caa1b02'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.350279 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a25fdfa9-fb18-4f9f-8491-031a4a52ac97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f29a27cffb8847f4a1111c308caa1b02'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.360391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:51.362889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a25fdfa9-fb18-4f9f-8491-031a4a52ac97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f29a27cffb8847f4a1111c308caa1b02'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.371070 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a25fdfa9-fb18-4f9f-8491-031a4a52ac97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f29a27cffb8847f4a1111c308caa1b02'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.376953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1150/2311] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:08:51 2022] DELETE /identity/v3/users/f29a27cffb8847f4a1111c308caa1b02 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:51.640187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:51.641611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:51.657058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda5696-16fc-4959-985e-730ae180eff8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:51.715183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e290b10-646e-4290-b42d-562a659eac21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:51.716081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e290b10-646e-4290-b42d-562a659eac21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:51.716366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e290b10-646e-4290-b42d-562a659eac21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:51.716682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e290b10-646e-4290-b42d-562a659eac21 admin admin] PATH_INFO: `/v3/projects/97aa82e640744950bd821680ba7eba4f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:51.717903 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e290b10-646e-4290-b42d-562a659eac21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97aa82e640744950bd821680ba7eba4f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:51.719860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e290b10-646e-4290-b42d-562a659eac21 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:51.765192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e290b10-646e-4290-b42d-562a659eac21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97aa82e640744950bd821680ba7eba4f', 'request_id': 'req-4e290b10-646e-4290-b42d-562a659eac21'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.771721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e290b10-646e-4290-b42d-562a659eac21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97aa82e640744950bd821680ba7eba4f', 'request_id': 'req-4e290b10-646e-4290-b42d-562a659eac21'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.777281 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e290b10-646e-4290-b42d-562a659eac21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97aa82e640744950bd821680ba7eba4f', 'request_id': 'req-4e290b10-646e-4290-b42d-562a659eac21'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.789364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e290b10-646e-4290-b42d-562a659eac21 admin admin] The token cache is being invalidate because project 97aa82e640744950bd821680ba7eba4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:08:51.789662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e290b10-646e-4290-b42d-562a659eac21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.790275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e290b10-646e-4290-b42d-562a659eac21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:51.791825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1162/2312] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:08:51 2022] DELETE /identity/v3/projects/97aa82e640744950bd821680ba7eba4f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:51.797659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a947684-4378-4158-a883-4ccd9d79cb4e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:51.798122 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a947684-4378-4158-a883-4ccd9d79cb4e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:51.798223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a947684-4378-4158-a883-4ccd9d79cb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:51.816698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2a947684-4378-4158-a883-4ccd9d79cb4e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:51.817836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2a947684-4378-4158-a883-4ccd9d79cb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:51.875717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1151/2313] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:51.884555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c390f-db74-4f14-8fc0-179464e39734 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:51.884850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c390f-db74-4f14-8fc0-179464e39734 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:51.885135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c390f-db74-4f14-8fc0-179464e39734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:51.903118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ad5c390f-db74-4f14-8fc0-179464e39734 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:51.904356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ad5c390f-db74-4f14-8fc0-179464e39734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:51.912328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1163/2314] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:51.916342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a947684-4378-4158-a883-4ccd9d79cb4e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:51.925999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a947684-4378-4158-a883-4ccd9d79cb4e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:51.931007 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1cac8-7335-42e9-aff2-27880d7dba7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:51.931674 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1cac8-7335-42e9-aff2-27880d7dba7d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:51.932028 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1cac8-7335-42e9-aff2-27880d7dba7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:51.932302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1cac8-7335-42e9-aff2-27880d7dba7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:51.933021 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1cac8-7335-42e9-aff2-27880d7dba7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:08:51.933327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1cac8-7335-42e9-aff2-27880d7dba7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:51.934905 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1cac8-7335-42e9-aff2-27880d7dba7d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:51.946789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1152/2315] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:08:51.951096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c390f-db74-4f14-8fc0-179464e39734 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:51.960273 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c390f-db74-4f14-8fc0-179464e39734 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:51.964821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7548baf-e40b-4819-882f-0f18c1262434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:51.965456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7548baf-e40b-4819-882f-0f18c1262434 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:51.965738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7548baf-e40b-4819-882f-0f18c1262434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:51.966015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7548baf-e40b-4819-882f-0f18c1262434 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:51.966698 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7548baf-e40b-4819-882f-0f18c1262434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:51.968675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7548baf-e40b-4819-882f-0f18c1262434 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:51.993708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1164/2316] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:08:51 2022] POST /identity/v3/projects => generated 383 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:08:51.997650 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1cac8-7335-42e9-aff2-27880d7dba7d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:52.008327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1cac8-7335-42e9-aff2-27880d7dba7d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:52.063473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2786b-a7e8-425e-bfb8-6804d039516b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:52.064155 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2786b-a7e8-425e-bfb8-6804d039516b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:52.064439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2786b-a7e8-425e-bfb8-6804d039516b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:52.064715 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2786b-a7e8-425e-bfb8-6804d039516b admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:52.065390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2786b-a7e8-425e-bfb8-6804d039516b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:52.067349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2786b-a7e8-425e-bfb8-6804d039516b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:52.089669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1153/2317] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:08:51 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:08:52.091152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7548baf-e40b-4819-882f-0f18c1262434 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:52.102663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7548baf-e40b-4819-882f-0f18c1262434 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:52.107628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb153b-3ef0-4065-af6d-3a88e25f6060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:52.108321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb153b-3ef0-4065-af6d-3a88e25f6060 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:52.108652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb153b-3ef0-4065-af6d-3a88e25f6060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:52.108933 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb153b-3ef0-4065-af6d-3a88e25f6060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:52.109708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb153b-3ef0-4065-af6d-3a88e25f6060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:52.111307 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb153b-3ef0-4065-af6d-3a88e25f6060 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:52.113546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:52.124715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1165/2318] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:08:52.128683 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2786b-a7e8-425e-bfb8-6804d039516b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:52.143153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2786b-a7e8-425e-bfb8-6804d039516b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:52.148197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a09184-26dd-49f7-a43a-be97c30777d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:52.148855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a09184-26dd-49f7-a43a-be97c30777d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:52.149135 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a09184-26dd-49f7-a43a-be97c30777d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:52.149409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a09184-26dd-49f7-a43a-be97c30777d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:52.150135 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a09184-26dd-49f7-a43a-be97c30777d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:52.151760 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a09184-26dd-49f7-a43a-be97c30777d8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:52.162423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1154/2319] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:08:52.166569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb153b-3ef0-4065-af6d-3a88e25f6060 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:52.176610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb153b-3ef0-4065-af6d-3a88e25f6060 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:52.181859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3c6d3-673c-4f6d-ab83-8a3b2b78e6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:52.182734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3c6d3-673c-4f6d-ab83-8a3b2b78e6a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:52.183046 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3c6d3-673c-4f6d-ab83-8a3b2b78e6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:52.183325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3c6d3-673c-4f6d-ab83-8a3b2b78e6a4 admin admin] PATH_INFO: `/v3/projects/d62ea7a9f3e94f02a9d66f5b86f2e0df/users/b0be7dcc0c694562b41935370d2b5533/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:52.199459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3c6d3-673c-4f6d-ab83-8a3b2b78e6a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62ea7a9f3e94f02a9d66f5b86f2e0df, user_id=b0be7dcc0c694562b41935370d2b5533, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:52.202148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3c6d3-673c-4f6d-ab83-8a3b2b78e6a4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:52.214729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1166/2320] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:08:52 2022] PUT /identity/v3/projects/d62ea7a9f3e94f02a9d66f5b86f2e0df/users/b0be7dcc0c694562b41935370d2b5533/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:52.227083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae769b-deb2-4438-a450-ef980b58589b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:52.227369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae769b-deb2-4438-a450-ef980b58589b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:52.227646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae769b-deb2-4438-a450-ef980b58589b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:52.253794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-19ae769b-deb2-4438-a450-ef980b58589b None None] MFA Rules not processed for user `b0be7dcc0c694562b41935370d2b5533`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:52.254982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-19ae769b-deb2-4438-a450-ef980b58589b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:52.294418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:52.319995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1155/2321] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:52.330978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3c6d3-673c-4f6d-ab83-8a3b2b78e6a4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:52.332687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e1a3c6d3-673c-4f6d-ab83-8a3b2b78e6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:52.388234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3c6d3-673c-4f6d-ab83-8a3b2b78e6a4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:52.391945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6affe2-8692-4954-bf4a-e13fbf662ede service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:52.392955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6affe2-8692-4954-bf4a-e13fbf662ede service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:52.393228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6affe2-8692-4954-bf4a-e13fbf662ede service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:52.393503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6affe2-8692-4954-bf4a-e13fbf662ede service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:52.401827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6affe2-8692-4954-bf4a-e13fbf662ede service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:52.403287 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6affe2-8692-4954-bf4a-e13fbf662ede service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:52.412621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1167/2322] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:52.431800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae769b-deb2-4438-a450-ef980b58589b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:52.493886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae769b-deb2-4438-a450-ef980b58589b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:52.498463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:52.499320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:52.499605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:52.499910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] PATH_INFO: `/v3/users/b0be7dcc0c694562b41935370d2b5533` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:52.501105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0be7dcc0c694562b41935370d2b5533)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:52.503012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:52.547020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0be7dcc0c694562b41935370d2b5533'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:52.547693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0be7dcc0c694562b41935370d2b5533'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:52.555288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0be7dcc0c694562b41935370d2b5533'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:52.567527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0be7dcc0c694562b41935370d2b5533'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:52.574261 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0be7dcc0c694562b41935370d2b5533'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:52.580064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1156/2323] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:08:52 2022] DELETE /identity/v3/users/b0be7dcc0c694562b41935370d2b5533 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:52.585007 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc83ed-cb7d-46b5-b3db-8d2298a1e724 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:52.585296 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc83ed-cb7d-46b5-b3db-8d2298a1e724 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:52.585571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc83ed-cb7d-46b5-b3db-8d2298a1e724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:52.602781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a4fc83ed-cb7d-46b5-b3db-8d2298a1e724 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:52.603988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a4fc83ed-cb7d-46b5-b3db-8d2298a1e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:52.659822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1168/2324] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:52.669863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:52.671285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:52.726826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b2600-57c1-4e8b-835c-23f90eaebe02 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:52.730442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e6b47-ed3a-4e34-b379-6c1c28cf2223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:52.731073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e6b47-ed3a-4e34-b379-6c1c28cf2223 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:52.731351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e6b47-ed3a-4e34-b379-6c1c28cf2223 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:52.731621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e6b47-ed3a-4e34-b379-6c1c28cf2223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:52.740486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e6b47-ed3a-4e34-b379-6c1c28cf2223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:52.741893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e6b47-ed3a-4e34-b379-6c1c28cf2223 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:52.751132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1157/2325] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:08:53.009064 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc83ed-cb7d-46b5-b3db-8d2298a1e724 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:53.010511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a4fc83ed-cb7d-46b5-b3db-8d2298a1e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:53.024518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc83ed-cb7d-46b5-b3db-8d2298a1e724 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:53.029108 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:53.030620 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:53.031253 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:53.031536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8 admin admin] PATH_INFO: `/v3/projects/d62ea7a9f3e94f02a9d66f5b86f2e0df` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:53.032787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d62ea7a9f3e94f02a9d66f5b86f2e0df)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:53.034697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:53.081568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62ea7a9f3e94f02a9d66f5b86f2e0df', 'request_id': 'req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:53.088484 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62ea7a9f3e94f02a9d66f5b86f2e0df', 'request_id': 'req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:53.094374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62ea7a9f3e94f02a9d66f5b86f2e0df', 'request_id': 'req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:53.103335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:53.107389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8 admin admin] The token cache is being invalidate because project d62ea7a9f3e94f02a9d66f5b86f2e0df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:08:53.107711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:53.108750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c1c9e6d8-be2f-4b62-96c3-8a39306044f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:08:53.110371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1169/2326] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:08:53 2022] DELETE /identity/v3/projects/d62ea7a9f3e94f02a9d66f5b86f2e0df => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:53.115696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0eeda-40a3-44f4-ba5d-f01ef5627d3b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:53.116181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0eeda-40a3-44f4-ba5d-f01ef5627d3b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:53.116288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0eeda-40a3-44f4-ba5d-f01ef5627d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:53.122530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:53.136263 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-67d0eeda-40a3-44f4-ba5d-f01ef5627d3b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:53.137730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-67d0eeda-40a3-44f4-ba5d-f01ef5627d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:53.195275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1158/2327] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:53.203116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d272f-5daf-4c71-89da-112b0493e908 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:53.203405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d272f-5daf-4c71-89da-112b0493e908 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:53.203724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d272f-5daf-4c71-89da-112b0493e908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:53.224253 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-805d272f-5daf-4c71-89da-112b0493e908 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:53.225511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-805d272f-5daf-4c71-89da-112b0493e908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:53.233807 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1170/2328] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:08:53.238057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0eeda-40a3-44f4-ba5d-f01ef5627d3b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:53.247836 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0eeda-40a3-44f4-ba5d-f01ef5627d3b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:53.253368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a431e4-4ceb-4589-b72a-e3f9a5987524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:53.254059 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a431e4-4ceb-4589-b72a-e3f9a5987524 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:53.254351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a431e4-4ceb-4589-b72a-e3f9a5987524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:53.254628 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a431e4-4ceb-4589-b72a-e3f9a5987524 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:53.256385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a431e4-4ceb-4589-b72a-e3f9a5987524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:08:53.256385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a431e4-4ceb-4589-b72a-e3f9a5987524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:53.257906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a431e4-4ceb-4589-b72a-e3f9a5987524 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:53.270996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1159/2329] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:08:53.275494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d272f-5daf-4c71-89da-112b0493e908 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:53.285246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d272f-5daf-4c71-89da-112b0493e908 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:53.290037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62eb68c-9e2d-4568-91e3-45ee31f62125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:53.290701 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eb68c-9e2d-4568-91e3-45ee31f62125 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:53.290987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eb68c-9e2d-4568-91e3-45ee31f62125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:53.292023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eb68c-9e2d-4568-91e3-45ee31f62125 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:53.292992 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62eb68c-9e2d-4568-91e3-45ee31f62125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:53.295008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62eb68c-9e2d-4568-91e3-45ee31f62125 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:53.322668 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1171/2330] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:08:53 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:08:53.326823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a431e4-4ceb-4589-b72a-e3f9a5987524 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:53.337335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a431e4-4ceb-4589-b72a-e3f9a5987524 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:53.397076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65e653-66e5-4889-8c4a-124fd5040d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:53.397660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65e653-66e5-4889-8c4a-124fd5040d70 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:53.397943 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65e653-66e5-4889-8c4a-124fd5040d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:53.398219 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65e653-66e5-4889-8c4a-124fd5040d70 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:53.398922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65e653-66e5-4889-8c4a-124fd5040d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:53.400899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65e653-66e5-4889-8c4a-124fd5040d70 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:53.421270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1160/2331] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:08:53 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:08:53.425418 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62eb68c-9e2d-4568-91e3-45ee31f62125 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:53.436663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62eb68c-9e2d-4568-91e3-45ee31f62125 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:53.441801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acea476-c6e9-46ae-831b-0c66a57f5d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:53.442541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acea476-c6e9-46ae-831b-0c66a57f5d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:53.442924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acea476-c6e9-46ae-831b-0c66a57f5d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:53.443950 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acea476-c6e9-46ae-831b-0c66a57f5d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:53.444754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acea476-c6e9-46ae-831b-0c66a57f5d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:53.446412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acea476-c6e9-46ae-831b-0c66a57f5d76 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:53.458106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1172/2332] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:08:53.462266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65e653-66e5-4889-8c4a-124fd5040d70 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:53.472441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65e653-66e5-4889-8c4a-124fd5040d70 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:53.477077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c43904-417f-4525-b7ad-91d2060e3e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:53.477726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c43904-417f-4525-b7ad-91d2060e3e58 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:53.478013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c43904-417f-4525-b7ad-91d2060e3e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:53.478294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c43904-417f-4525-b7ad-91d2060e3e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:53.479025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c43904-417f-4525-b7ad-91d2060e3e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:53.480750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c43904-417f-4525-b7ad-91d2060e3e58 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:53.491012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1161/2333] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:08:53.494940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acea476-c6e9-46ae-831b-0c66a57f5d76 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:53.504760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acea476-c6e9-46ae-831b-0c66a57f5d76 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:53.509860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a933af8-a813-4383-9841-8a072be6a3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:53.510722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a933af8-a813-4383-9841-8a072be6a3ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:53.510999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a933af8-a813-4383-9841-8a072be6a3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:53.511275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a933af8-a813-4383-9841-8a072be6a3ee admin admin] PATH_INFO: `/v3/projects/99bf49f935c2471c91b9a5d42b9b26a9/users/5e7300fc644f442fbbbd6eefc5706247/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:53.525688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a933af8-a813-4383-9841-8a072be6a3ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=99bf49f935c2471c91b9a5d42b9b26a9, user_id=5e7300fc644f442fbbbd6eefc5706247, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:53.528375 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a933af8-a813-4383-9841-8a072be6a3ee admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:53.540855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1173/2334] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:08:53 2022] PUT /identity/v3/projects/99bf49f935c2471c91b9a5d42b9b26a9/users/5e7300fc644f442fbbbd6eefc5706247/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:08:53.550876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169d643-3ee4-47e6-8d30-2f74bd6c1ccb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:53.551198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169d643-3ee4-47e6-8d30-2f74bd6c1ccb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:53.551479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169d643-3ee4-47e6-8d30-2f74bd6c1ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:53.578460 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b169d643-3ee4-47e6-8d30-2f74bd6c1ccb None None] MFA Rules not processed for user `5e7300fc644f442fbbbd6eefc5706247`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:53.579697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b169d643-3ee4-47e6-8d30-2f74bd6c1ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:53.638188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:53.644566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1162/2335] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:08:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:53.655577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a933af8-a813-4383-9841-8a072be6a3ee admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:53.657384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9a933af8-a813-4383-9841-8a072be6a3ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:53.712720 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a933af8-a813-4383-9841-8a072be6a3ee admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:53.716557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f6daf-575e-425a-aaf1-876c1ba12236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:53.717192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f6daf-575e-425a-aaf1-876c1ba12236 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:53.717470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f6daf-575e-425a-aaf1-876c1ba12236 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:53.717743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f6daf-575e-425a-aaf1-876c1ba12236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:53.726355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f6daf-575e-425a-aaf1-876c1ba12236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:53.727812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f6daf-575e-425a-aaf1-876c1ba12236 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:53.736365 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1174/2336] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:09:03.648245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:03.732428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:04.049139 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169d643-3ee4-47e6-8d30-2f74bd6c1ccb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:04.050565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b169d643-3ee4-47e6-8d30-2f74bd6c1ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:04.123020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169d643-3ee4-47e6-8d30-2f74bd6c1ccb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:04.128525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02ee66-12e9-4913-b911-fd57c0dad265 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:04.129180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02ee66-12e9-4913-b911-fd57c0dad265 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:04.129439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02ee66-12e9-4913-b911-fd57c0dad265 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:04.129710 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02ee66-12e9-4913-b911-fd57c0dad265 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:04.131472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bb02ee66-12e9-4913-b911-fd57c0dad265 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:04.188800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02ee66-12e9-4913-b911-fd57c0dad265 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:04.190451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02ee66-12e9-4913-b911-fd57c0dad265 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:04.200209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1163/2337] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:09:13.209727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f6daf-575e-425a-aaf1-876c1ba12236 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:13.274189 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f6daf-575e-425a-aaf1-876c1ba12236 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:13.278803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:13.279700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:13.279989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:13.280270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] PATH_INFO: `/v3/users/5e7300fc644f442fbbbd6eefc5706247` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:13.281503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e7300fc644f442fbbbd6eefc5706247)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:13.283424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:13.329289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e7300fc644f442fbbbd6eefc5706247'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:13.329963 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e7300fc644f442fbbbd6eefc5706247'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:13.330589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e7300fc644f442fbbbd6eefc5706247'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:13.344448 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e7300fc644f442fbbbd6eefc5706247'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:13.351546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e7300fc644f442fbbbd6eefc5706247'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:13.366309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1175/2338] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:09:13 2022] DELETE /identity/v3/users/5e7300fc644f442fbbbd6eefc5706247 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:13.370947 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb40d27-6147-4178-89e1-ec2d126bb2fb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:13.371442 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb40d27-6147-4178-89e1-ec2d126bb2fb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:13.371546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb40d27-6147-4178-89e1-ec2d126bb2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:13.390896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ffb40d27-6147-4178-89e1-ec2d126bb2fb None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:13.392142 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ffb40d27-6147-4178-89e1-ec2d126bb2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:13.463839 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1164/2339] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:13.470146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:13.471894 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:13.533467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d068d7d-4fb2-4a0c-99f0-9d7eebc3bcb0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:13.537422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daef58-24be-4416-b715-73d982e7b63f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:13.538105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daef58-24be-4416-b715-73d982e7b63f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:13.538395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daef58-24be-4416-b715-73d982e7b63f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:13.538682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daef58-24be-4416-b715-73d982e7b63f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:13.548531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52daef58-24be-4416-b715-73d982e7b63f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:13.549977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52daef58-24be-4416-b715-73d982e7b63f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:13.560501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1176/2340] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:09:13.853535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb40d27-6147-4178-89e1-ec2d126bb2fb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:13.854976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ffb40d27-6147-4178-89e1-ec2d126bb2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:13.867166 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb40d27-6147-4178-89e1-ec2d126bb2fb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:13.872100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:13.872979 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:13.873263 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:13.873545 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3 admin admin] PATH_INFO: `/v3/projects/99bf49f935c2471c91b9a5d42b9b26a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:13.874746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99bf49f935c2471c91b9a5d42b9b26a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:13.876747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:13.913848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:13.927171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99bf49f935c2471c91b9a5d42b9b26a9', 'request_id': 'req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:13.934230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99bf49f935c2471c91b9a5d42b9b26a9', 'request_id': 'req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:13.940054 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99bf49f935c2471c91b9a5d42b9b26a9', 'request_id': 'req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:13.951525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3 admin admin] The token cache is being invalidate because project 99bf49f935c2471c91b9a5d42b9b26a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:09:13.951844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:13.952472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a0de6e6e-9b12-4e81-a31b-9ce0a9abf8a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:13.953996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1165/2341] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:09:13 2022] DELETE /identity/v3/projects/99bf49f935c2471c91b9a5d42b9b26a9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:13.958867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aea5b-71ae-4f41-a4b9-a7ff61e6a441 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:13.959158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aea5b-71ae-4f41-a4b9-a7ff61e6a441 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:13.959437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aea5b-71ae-4f41-a4b9-a7ff61e6a441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:13.977687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9e1aea5b-71ae-4f41-a4b9-a7ff61e6a441 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:13.978891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9e1aea5b-71ae-4f41-a4b9-a7ff61e6a441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:13.996184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:14.039743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1177/2342] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:14.047701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dada95b-2f37-4bf2-9241-d52f663f2b0a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:14.048004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dada95b-2f37-4bf2-9241-d52f663f2b0a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:14.048281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dada95b-2f37-4bf2-9241-d52f663f2b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:14.066033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6dada95b-2f37-4bf2-9241-d52f663f2b0a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:14.067241 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6dada95b-2f37-4bf2-9241-d52f663f2b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:14.075623 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1166/2343] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:14.079861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aea5b-71ae-4f41-a4b9-a7ff61e6a441 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:14.090148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aea5b-71ae-4f41-a4b9-a7ff61e6a441 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:14.094751 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b093add-4afb-4ebd-a0c7-50e0e2c41dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:14.095409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b093add-4afb-4ebd-a0c7-50e0e2c41dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:14.095747 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b093add-4afb-4ebd-a0c7-50e0e2c41dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:14.096015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b093add-4afb-4ebd-a0c7-50e0e2c41dc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:14.096777 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b093add-4afb-4ebd-a0c7-50e0e2c41dc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:09:14.097098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b093add-4afb-4ebd-a0c7-50e0e2c41dc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:14.098707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b093add-4afb-4ebd-a0c7-50e0e2c41dc8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:14.110544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1178/2344] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:09:14.114847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dada95b-2f37-4bf2-9241-d52f663f2b0a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:14.124180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dada95b-2f37-4bf2-9241-d52f663f2b0a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:14.129649 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e6a16-3d92-4fa9-a948-33277c246d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:14.130315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e6a16-3d92-4fa9-a948-33277c246d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:14.130597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e6a16-3d92-4fa9-a948-33277c246d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:14.130889 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e6a16-3d92-4fa9-a948-33277c246d40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:14.131599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e6a16-3d92-4fa9-a948-33277c246d40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:14.134206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e6a16-3d92-4fa9-a948-33277c246d40 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:14.159556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1167/2345] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:09:14 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:09:14.163783 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b093add-4afb-4ebd-a0c7-50e0e2c41dc8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:14.173689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b093add-4afb-4ebd-a0c7-50e0e2c41dc8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:14.233091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf2ae2-9f44-4899-8e5a-c8d9df1d7647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:14.233731 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf2ae2-9f44-4899-8e5a-c8d9df1d7647 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:14.233949 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf2ae2-9f44-4899-8e5a-c8d9df1d7647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:14.234232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf2ae2-9f44-4899-8e5a-c8d9df1d7647 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:14.234921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf2ae2-9f44-4899-8e5a-c8d9df1d7647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:14.236877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf2ae2-9f44-4899-8e5a-c8d9df1d7647 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:14.255773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1179/2346] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:09:14 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:14.260499 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e6a16-3d92-4fa9-a948-33277c246d40 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:14.270907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e6a16-3d92-4fa9-a948-33277c246d40 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:14.275588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11c92d-b376-4b98-a777-5c034e17ada3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:14.276283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11c92d-b376-4b98-a777-5c034e17ada3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:14.276569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11c92d-b376-4b98-a777-5c034e17ada3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:14.276846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11c92d-b376-4b98-a777-5c034e17ada3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:14.277567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11c92d-b376-4b98-a777-5c034e17ada3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:14.279151 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11c92d-b376-4b98-a777-5c034e17ada3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:14.289452 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1168/2347] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:14.293258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf2ae2-9f44-4899-8e5a-c8d9df1d7647 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:14.303084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf2ae2-9f44-4899-8e5a-c8d9df1d7647 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:14.308184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ddaa8-8897-48fb-a3ca-66ecdad2f411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:14.308830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ddaa8-8897-48fb-a3ca-66ecdad2f411 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:14.309114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ddaa8-8897-48fb-a3ca-66ecdad2f411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:14.309389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ddaa8-8897-48fb-a3ca-66ecdad2f411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:14.310119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ddaa8-8897-48fb-a3ca-66ecdad2f411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:14.311730 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ddaa8-8897-48fb-a3ca-66ecdad2f411 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:14.322078 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1180/2348] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:14.325869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11c92d-b376-4b98-a777-5c034e17ada3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:14.335739 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11c92d-b376-4b98-a777-5c034e17ada3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:14.340262 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cc6aa-c02e-4a65-938b-92aca9289a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:14.341130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cc6aa-c02e-4a65-938b-92aca9289a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:14.341418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cc6aa-c02e-4a65-938b-92aca9289a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:14.341698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cc6aa-c02e-4a65-938b-92aca9289a17 admin admin] PATH_INFO: `/v3/projects/5602b9db0a89433894a93fe39df5f51e/users/61cb43c95c99413c852ef62ea41de950/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:14.356390 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cc6aa-c02e-4a65-938b-92aca9289a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5602b9db0a89433894a93fe39df5f51e, user_id=61cb43c95c99413c852ef62ea41de950, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:14.359016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cc6aa-c02e-4a65-938b-92aca9289a17 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:14.373873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1169/2349] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:14 2022] PUT /identity/v3/projects/5602b9db0a89433894a93fe39df5f51e/users/61cb43c95c99413c852ef62ea41de950/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:14.384695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700a644-c275-4a48-b3a4-b689aa554bf4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:14.384987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700a644-c275-4a48-b3a4-b689aa554bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:14.385269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700a644-c275-4a48-b3a4-b689aa554bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:14.411833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0700a644-c275-4a48-b3a4-b689aa554bf4 None None] MFA Rules not processed for user `61cb43c95c99413c852ef62ea41de950`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:14.413050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0700a644-c275-4a48-b3a4-b689aa554bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:14.475892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1181/2350] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:14.487316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cc6aa-c02e-4a65-938b-92aca9289a17 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:14.488982 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-704cc6aa-c02e-4a65-938b-92aca9289a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:14.544970 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cc6aa-c02e-4a65-938b-92aca9289a17 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:14.548597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94719ca5-1643-4a07-8c5c-22c894b99789 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af6cc8c56db48beae2550954446c119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:14.549237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94719ca5-1643-4a07-8c5c-22c894b99789 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:14.549524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94719ca5-1643-4a07-8c5c-22c894b99789 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:14.549805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94719ca5-1643-4a07-8c5c-22c894b99789 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:14.558628 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94719ca5-1643-4a07-8c5c-22c894b99789 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:14.560081 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94719ca5-1643-4a07-8c5c-22c894b99789 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:14.568860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1170/2351] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:09:24.472970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:24.564893 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:25.113153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab394a-2537-4eb9-9ee4-5c887c6acf41 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:25.113578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab394a-2537-4eb9-9ee4-5c887c6acf41 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:25.113772 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab394a-2537-4eb9-9ee4-5c887c6acf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:25.133548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-74ab394a-2537-4eb9-9ee4-5c887c6acf41 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:25.135305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-74ab394a-2537-4eb9-9ee4-5c887c6acf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:25.203038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1182/2352] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:09:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:25.253349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94719ca5-1643-4a07-8c5c-22c894b99789 service glance] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:25.255217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-94719ca5-1643-4a07-8c5c-22c894b99789 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:25.341118 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94719ca5-1643-4a07-8c5c-22c894b99789 service glance] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:25.356497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec3b5e-cd72-4787-9897-0b0c0980ed23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:25.356497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec3b5e-cd72-4787-9897-0b0c0980ed23 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:25.356497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec3b5e-cd72-4787-9897-0b0c0980ed23 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:25.356497 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec3b5e-cd72-4787-9897-0b0c0980ed23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:25.359298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec3b5e-cd72-4787-9897-0b0c0980ed23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:25.361028 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec3b5e-cd72-4787-9897-0b0c0980ed23 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:25.376005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1171/2353] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:09:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:09:35.195612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=25 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:35.371587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=32 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:38.084280 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab394a-2537-4eb9-9ee4-5c887c6acf41 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:38.085700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-74ab394a-2537-4eb9-9ee4-5c887c6acf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:38.145637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab394a-2537-4eb9-9ee4-5c887c6acf41 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:38.149276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeff78c-6850-44ab-9135-30c15e63af04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:38.149915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff78c-6850-44ab-9135-30c15e63af04 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:38.150195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff78c-6850-44ab-9135-30c15e63af04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:38.150468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff78c-6850-44ab-9135-30c15e63af04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:38.152239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0eeff78c-6850-44ab-9135-30c15e63af04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:38.213681 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeff78c-6850-44ab-9135-30c15e63af04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:38.215110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeff78c-6850-44ab-9135-30c15e63af04 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:38.224661 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1183/2354] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:09:41.262148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec3b5e-cd72-4787-9897-0b0c0980ed23 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:41.263229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-38ec3b5e-cd72-4787-9897-0b0c0980ed23 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:41.339446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeff78c-6850-44ab-9135-30c15e63af04 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:41.341056 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0eeff78c-6850-44ab-9135-30c15e63af04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:41.358829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeff78c-6850-44ab-9135-30c15e63af04 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:41.364407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00ad9e-6a13-46e4-904f-a81fb4da00c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:41.365470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00ad9e-6a13-46e4-904f-a81fb4da00c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:41.365778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00ad9e-6a13-46e4-904f-a81fb4da00c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:41.366058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00ad9e-6a13-46e4-904f-a81fb4da00c3 admin admin] PATH_INFO: `/v3/users/303e8aae1bd04228bedb145dec891a3e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:41.367282 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00ad9e-6a13-46e4-904f-a81fb4da00c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=303e8aae1bd04228bedb145dec891a3e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:41.369831 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00ad9e-6a13-46e4-904f-a81fb4da00c3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:41.422367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec3b5e-cd72-4787-9897-0b0c0980ed23 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:41.433784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa24e3-092d-4e65-88fd-248b4c7b89c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:41.435381 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa24e3-092d-4e65-88fd-248b4c7b89c5 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:41.435894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa24e3-092d-4e65-88fd-248b4c7b89c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:41.436396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa24e3-092d-4e65-88fd-248b4c7b89c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:41.438706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-77fa24e3-092d-4e65-88fd-248b4c7b89c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:41.453304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa24e3-092d-4e65-88fd-248b4c7b89c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:41.455737 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa24e3-092d-4e65-88fd-248b4c7b89c5 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:41.466566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9b00ad9e-6a13-46e4-904f-a81fb4da00c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303e8aae1bd04228bedb145dec891a3e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:41.467534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9b00ad9e-6a13-46e4-904f-a81fb4da00c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303e8aae1bd04228bedb145dec891a3e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:41.468374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9b00ad9e-6a13-46e4-904f-a81fb4da00c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303e8aae1bd04228bedb145dec891a3e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:41.480065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1172/2355] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:09:41.493670 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9b00ad9e-6a13-46e4-904f-a81fb4da00c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303e8aae1bd04228bedb145dec891a3e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:41.508509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9b00ad9e-6a13-46e4-904f-a81fb4da00c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303e8aae1bd04228bedb145dec891a3e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:41.529187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1184/2356] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:09:41 2022] DELETE /identity/v3/users/303e8aae1bd04228bedb145dec891a3e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:09:41.739194 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa24e3-092d-4e65-88fd-248b4c7b89c5 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:41.753805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa24e3-092d-4e65-88fd-248b4c7b89c5 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:41.824434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bf699-d748-47c7-a8ed-e73008d90380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:41.825316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bf699-d748-47c7-a8ed-e73008d90380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:41.825577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bf699-d748-47c7-a8ed-e73008d90380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:41.825851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bf699-d748-47c7-a8ed-e73008d90380 admin admin] PATH_INFO: `/v3/projects/ac9f9313c3ac4dde97e75ac6fa8fae11` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:41.827056 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bf699-d748-47c7-a8ed-e73008d90380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9f9313c3ac4dde97e75ac6fa8fae11)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:41.829412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bf699-d748-47c7-a8ed-e73008d90380 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:41.877519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ac1bf699-d748-47c7-a8ed-e73008d90380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9f9313c3ac4dde97e75ac6fa8fae11', 'request_id': 'req-ac1bf699-d748-47c7-a8ed-e73008d90380'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:41.886406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ac1bf699-d748-47c7-a8ed-e73008d90380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9f9313c3ac4dde97e75ac6fa8fae11', 'request_id': 'req-ac1bf699-d748-47c7-a8ed-e73008d90380'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:41.893071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ac1bf699-d748-47c7-a8ed-e73008d90380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9f9313c3ac4dde97e75ac6fa8fae11', 'request_id': 'req-ac1bf699-d748-47c7-a8ed-e73008d90380'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:41.905832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ac1bf699-d748-47c7-a8ed-e73008d90380 admin admin] The token cache is being invalidate because project ac9f9313c3ac4dde97e75ac6fa8fae11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:09:41.906213 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ac1bf699-d748-47c7-a8ed-e73008d90380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:41.906823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ac1bf699-d748-47c7-a8ed-e73008d90380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:41.908444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1173/2357] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:09:41 2022] DELETE /identity/v3/projects/ac9f9313c3ac4dde97e75ac6fa8fae11 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:41.913783 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e8065-0695-4042-913b-b6b16093783a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:41.914074 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e8065-0695-4042-913b-b6b16093783a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:41.914383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e8065-0695-4042-913b-b6b16093783a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:41.933591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fe8e8065-0695-4042-913b-b6b16093783a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:41.934806 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fe8e8065-0695-4042-913b-b6b16093783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:41.952661 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:42.002131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1185/2358] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:42.010817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46da202-af2d-4847-96ce-b40587d33c95 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:42.011107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46da202-af2d-4847-96ce-b40587d33c95 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:42.011381 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46da202-af2d-4847-96ce-b40587d33c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:42.033922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b46da202-af2d-4847-96ce-b40587d33c95 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:42.035148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b46da202-af2d-4847-96ce-b40587d33c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:42.043314 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1174/2359] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:42.047571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e8065-0695-4042-913b-b6b16093783a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:42.058057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e8065-0695-4042-913b-b6b16093783a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:42.062581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c50682-65bf-4441-b3ec-ec7f0510c4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:42.063236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c50682-65bf-4441-b3ec-ec7f0510c4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:42.063517 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c50682-65bf-4441-b3ec-ec7f0510c4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:42.063830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c50682-65bf-4441-b3ec-ec7f0510c4f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:42.064555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c50682-65bf-4441-b3ec-ec7f0510c4f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:09:42.064863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c50682-65bf-4441-b3ec-ec7f0510c4f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:42.066460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c50682-65bf-4441-b3ec-ec7f0510c4f7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:42.078798 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1186/2360] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:09:42.083143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46da202-af2d-4847-96ce-b40587d33c95 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:42.093263 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46da202-af2d-4847-96ce-b40587d33c95 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:42.098371 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e53549-c5cc-4d1d-beb0-180208a613d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:42.098955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e53549-c5cc-4d1d-beb0-180208a613d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:42.099238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e53549-c5cc-4d1d-beb0-180208a613d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:42.099515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e53549-c5cc-4d1d-beb0-180208a613d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:42.100434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e53549-c5cc-4d1d-beb0-180208a613d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:42.102339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e53549-c5cc-4d1d-beb0-180208a613d4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:42.113520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:42.135705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1175/2361] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:09:42 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:42.136976 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c50682-65bf-4441-b3ec-ec7f0510c4f7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:42.147623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c50682-65bf-4441-b3ec-ec7f0510c4f7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:42.205003 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea2f7a-c372-418a-aef1-01c2a0724851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:42.205607 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea2f7a-c372-418a-aef1-01c2a0724851 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:42.205890 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea2f7a-c372-418a-aef1-01c2a0724851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:42.206170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea2f7a-c372-418a-aef1-01c2a0724851 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:42.206857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea2f7a-c372-418a-aef1-01c2a0724851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:42.208805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea2f7a-c372-418a-aef1-01c2a0724851 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:42.227810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1187/2362] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:09:42 2022] POST /identity/v3/users => generated 302 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:42.232210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e53549-c5cc-4d1d-beb0-180208a613d4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:42.242323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e53549-c5cc-4d1d-beb0-180208a613d4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:42.247404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15a991-64e5-48c2-9911-66eaa7cce7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:42.248074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15a991-64e5-48c2-9911-66eaa7cce7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:42.248359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15a991-64e5-48c2-9911-66eaa7cce7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:42.248634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15a991-64e5-48c2-9911-66eaa7cce7f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:42.249354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15a991-64e5-48c2-9911-66eaa7cce7f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:42.251072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15a991-64e5-48c2-9911-66eaa7cce7f5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:42.261530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1176/2363] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:42.265738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea2f7a-c372-418a-aef1-01c2a0724851 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:42.275868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea2f7a-c372-418a-aef1-01c2a0724851 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:42.280419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7877f-d4d2-46d5-9bf2-c8289ff1c6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:42.281068 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7877f-d4d2-46d5-9bf2-c8289ff1c6ae admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:42.281351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7877f-d4d2-46d5-9bf2-c8289ff1c6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:42.281796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7877f-d4d2-46d5-9bf2-c8289ff1c6ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:42.282354 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7877f-d4d2-46d5-9bf2-c8289ff1c6ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:42.284065 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7877f-d4d2-46d5-9bf2-c8289ff1c6ae admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:42.294204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1188/2364] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:42.298170 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15a991-64e5-48c2-9911-66eaa7cce7f5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:42.308736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15a991-64e5-48c2-9911-66eaa7cce7f5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:42.313430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a901c-0bb8-4af0-8f7b-5e4ebe726902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:42.314380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a901c-0bb8-4af0-8f7b-5e4ebe726902 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:42.314564 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a901c-0bb8-4af0-8f7b-5e4ebe726902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:42.314835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a901c-0bb8-4af0-8f7b-5e4ebe726902 admin admin] PATH_INFO: `/v3/projects/eb758cbe277847529945b2d633c6f860/users/a59e0e7dc73e4d4783cb990cc6e4823b/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:42.330328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a901c-0bb8-4af0-8f7b-5e4ebe726902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb758cbe277847529945b2d633c6f860, user_id=a59e0e7dc73e4d4783cb990cc6e4823b, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:42.333077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a901c-0bb8-4af0-8f7b-5e4ebe726902 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:42.345807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1177/2365] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:42 2022] PUT /identity/v3/projects/eb758cbe277847529945b2d633c6f860/users/a59e0e7dc73e4d4783cb990cc6e4823b/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:09:42.350829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e8e34-17c2-40fd-abff-48e80c31492e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:42.351336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e8e34-17c2-40fd-abff-48e80c31492e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:42.351516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e8e34-17c2-40fd-abff-48e80c31492e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:42.371999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4e6e8e34-17c2-40fd-abff-48e80c31492e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:42.373212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4e6e8e34-17c2-40fd-abff-48e80c31492e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:42.436652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1189/2366] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:09:42.445992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a901c-0bb8-4af0-8f7b-5e4ebe726902 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:42.447732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7e4a901c-0bb8-4af0-8f7b-5e4ebe726902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:42.505809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a901c-0bb8-4af0-8f7b-5e4ebe726902 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:42.509518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228d27d-a246-4936-80d4-333c207276af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:42.510192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228d27d-a246-4936-80d4-333c207276af service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:42.510480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228d27d-a246-4936-80d4-333c207276af service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:42.510760 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228d27d-a246-4936-80d4-333c207276af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:42.520336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228d27d-a246-4936-80d4-333c207276af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:42.522751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228d27d-a246-4936-80d4-333c207276af service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:42.531740 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1178/2367] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:09:43.219882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9beeeb-11a3-4d7b-a0d5-d6e2eae5b67d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:43.220172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9beeeb-11a3-4d7b-a0d5-d6e2eae5b67d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:43.220451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9beeeb-11a3-4d7b-a0d5-d6e2eae5b67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:43.247110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ad9beeeb-11a3-4d7b-a0d5-d6e2eae5b67d None None] MFA Rules not processed for user `a59e0e7dc73e4d4783cb990cc6e4823b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:43.248341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ad9beeeb-11a3-4d7b-a0d5-d6e2eae5b67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:43.266031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:43.317030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1190/2368] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:43.321601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228d27d-a246-4936-80d4-333c207276af service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:43.331468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228d27d-a246-4936-80d4-333c207276af service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:43.335991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae00589-e33f-4c1a-ba63-502d90b3e3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:43.336637 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae00589-e33f-4c1a-ba63-502d90b3e3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:43.336919 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae00589-e33f-4c1a-ba63-502d90b3e3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:43.337344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae00589-e33f-4c1a-ba63-502d90b3e3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:43.338074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae00589-e33f-4c1a-ba63-502d90b3e3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:43.340058 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae00589-e33f-4c1a-ba63-502d90b3e3e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:43.368255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1179/2369] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:09:43 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:43.372642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9beeeb-11a3-4d7b-a0d5-d6e2eae5b67d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:43.383963 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9beeeb-11a3-4d7b-a0d5-d6e2eae5b67d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:43.442289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729eadf9-6233-4e36-88e8-04417e07312e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:43.442960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729eadf9-6233-4e36-88e8-04417e07312e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:43.443243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729eadf9-6233-4e36-88e8-04417e07312e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:43.443520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729eadf9-6233-4e36-88e8-04417e07312e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:43.444256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729eadf9-6233-4e36-88e8-04417e07312e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:43.447293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729eadf9-6233-4e36-88e8-04417e07312e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:43.471631 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1191/2370] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:09:43 2022] POST /identity/v3/users => generated 308 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:09:43.478097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae00589-e33f-4c1a-ba63-502d90b3e3e0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:43.496965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae00589-e33f-4c1a-ba63-502d90b3e3e0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:43.501805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5960a7-438e-4146-a0d5-fde8eaf56652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:43.502592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5960a7-438e-4146-a0d5-fde8eaf56652 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:43.502812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5960a7-438e-4146-a0d5-fde8eaf56652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:43.503062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5960a7-438e-4146-a0d5-fde8eaf56652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:43.503758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5960a7-438e-4146-a0d5-fde8eaf56652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:43.505361 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5960a7-438e-4146-a0d5-fde8eaf56652 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:43.521631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1180/2371] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:09:43.522560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729eadf9-6233-4e36-88e8-04417e07312e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:43.536953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729eadf9-6233-4e36-88e8-04417e07312e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:43.563574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9f69b-f31e-4188-96d3-c6601f8018e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:43.563574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f69b-f31e-4188-96d3-c6601f8018e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:43.563848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f69b-f31e-4188-96d3-c6601f8018e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:43.564098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f69b-f31e-4188-96d3-c6601f8018e5 admin admin] PATH_INFO: `/v3/projects/f29d3dc61c23486ba2aa70d92a724796/users/bda58cf35b384cc38b2d83e1a02b43f0/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:43.599912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e9f69b-f31e-4188-96d3-c6601f8018e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f29d3dc61c23486ba2aa70d92a724796, user_id=bda58cf35b384cc38b2d83e1a02b43f0, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:43.602700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e9f69b-f31e-4188-96d3-c6601f8018e5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:43.617577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1192/2372] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:43 2022] PUT /identity/v3/projects/f29d3dc61c23486ba2aa70d92a724796/users/bda58cf35b384cc38b2d83e1a02b43f0/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:43.622254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5960a7-438e-4146-a0d5-fde8eaf56652 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:43.635771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5960a7-438e-4146-a0d5-fde8eaf56652 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:43.696418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653f8a7-1521-4a49-afd7-99520e8f2463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:43.697097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653f8a7-1521-4a49-afd7-99520e8f2463 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:43.697954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653f8a7-1521-4a49-afd7-99520e8f2463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:43.698238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653f8a7-1521-4a49-afd7-99520e8f2463 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:43.699099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653f8a7-1521-4a49-afd7-99520e8f2463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:43.701041 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653f8a7-1521-4a49-afd7-99520e8f2463 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:43.715687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1181/2373] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:43.717988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9f69b-f31e-4188-96d3-c6601f8018e5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:43.730061 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9f69b-f31e-4188-96d3-c6601f8018e5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:43.736945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca28be-464e-458c-8cf6-1ccb9ca75fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:43.737989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca28be-464e-458c-8cf6-1ccb9ca75fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:43.738267 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca28be-464e-458c-8cf6-1ccb9ca75fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:43.738552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca28be-464e-458c-8cf6-1ccb9ca75fd9 admin admin] PATH_INFO: `/v3/projects/f29d3dc61c23486ba2aa70d92a724796/users/bda58cf35b384cc38b2d83e1a02b43f0/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:43.740650 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca28be-464e-458c-8cf6-1ccb9ca75fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f29d3dc61c23486ba2aa70d92a724796, user_id=bda58cf35b384cc38b2d83e1a02b43f0, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:43.743371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca28be-464e-458c-8cf6-1ccb9ca75fd9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:43.755029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1193/2374] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:43 2022] PUT /identity/v3/projects/f29d3dc61c23486ba2aa70d92a724796/users/bda58cf35b384cc38b2d83e1a02b43f0/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:44.795854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37718e-c481-49f9-85c4-91fe77636591 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:44.796809 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37718e-c481-49f9-85c4-91fe77636591 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:44.797381 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37718e-c481-49f9-85c4-91fe77636591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:44.833495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cc37718e-c481-49f9-85c4-91fe77636591 None None] MFA Rules not processed for user `bda58cf35b384cc38b2d83e1a02b43f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:44.835011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cc37718e-c481-49f9-85c4-91fe77636591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:44.847385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:44.922673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1182/2375] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:44 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:45.566309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca28be-464e-458c-8cf6-1ccb9ca75fd9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:45.569301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-51ca28be-464e-458c-8cf6-1ccb9ca75fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:45.669165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca28be-464e-458c-8cf6-1ccb9ca75fd9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:45.674445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8587ef5-97a8-4120-a257-f0e9ff914507 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:45.675099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587ef5-97a8-4120-a257-f0e9ff914507 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:45.675385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587ef5-97a8-4120-a257-f0e9ff914507 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:45.675675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587ef5-97a8-4120-a257-f0e9ff914507 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:45.677498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a8587ef5-97a8-4120-a257-f0e9ff914507 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:45.690286 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8587ef5-97a8-4120-a257-f0e9ff914507 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:45.691918 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8587ef5-97a8-4120-a257-f0e9ff914507 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:45.698447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:45.704076 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1194/2376] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:09:46.571026 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37718e-c481-49f9-85c4-91fe77636591 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:46.651969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37718e-c481-49f9-85c4-91fe77636591 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:46.656815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56845c52-21f8-4d59-baad-037f69052c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:46.657700 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56845c52-21f8-4d59-baad-037f69052c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:46.657981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56845c52-21f8-4d59-baad-037f69052c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:46.658667 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56845c52-21f8-4d59-baad-037f69052c61 admin admin] PATH_INFO: `/v3/users/a59e0e7dc73e4d4783cb990cc6e4823b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:46.659969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56845c52-21f8-4d59-baad-037f69052c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a59e0e7dc73e4d4783cb990cc6e4823b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:46.662255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56845c52-21f8-4d59-baad-037f69052c61 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:46.725544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-56845c52-21f8-4d59-baad-037f69052c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a59e0e7dc73e4d4783cb990cc6e4823b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:46.726299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-56845c52-21f8-4d59-baad-037f69052c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a59e0e7dc73e4d4783cb990cc6e4823b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:46.737661 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-56845c52-21f8-4d59-baad-037f69052c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a59e0e7dc73e4d4783cb990cc6e4823b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:46.756280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-56845c52-21f8-4d59-baad-037f69052c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a59e0e7dc73e4d4783cb990cc6e4823b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:46.767712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-56845c52-21f8-4d59-baad-037f69052c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a59e0e7dc73e4d4783cb990cc6e4823b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:46.779031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1183/2377] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:09:46 2022] DELETE /identity/v3/users/a59e0e7dc73e4d4783cb990cc6e4823b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:47.013321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8587ef5-97a8-4120-a257-f0e9ff914507 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:47.029992 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8587ef5-97a8-4120-a257-f0e9ff914507 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:47.121765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b0d28-f052-4293-92c5-da88b21acfe2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:47.122060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b0d28-f052-4293-92c5-da88b21acfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:47.122337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b0d28-f052-4293-92c5-da88b21acfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:47.140898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:47.141720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d89b0d28-f052-4293-92c5-da88b21acfe2 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:47.142078 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:47.142395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:47.142668 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] PATH_INFO: `/v3/projects/eb758cbe277847529945b2d633c6f860` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:47.142941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d89b0d28-f052-4293-92c5-da88b21acfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:47.143903 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb758cbe277847529945b2d633c6f860)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:47.145847 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:47.165735 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:47.201049 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb758cbe277847529945b2d633c6f860', 'request_id': 'req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:47.207560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb758cbe277847529945b2d633c6f860', 'request_id': 'req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:47.210009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1184/2378] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:09:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:47.216910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb758cbe277847529945b2d633c6f860', 'request_id': 'req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:47.224195 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b0d28-f052-4293-92c5-da88b21acfe2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:47.226217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d89b0d28-f052-4293-92c5-da88b21acfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:47.232638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] The token cache is being invalidate because project eb758cbe277847529945b2d633c6f860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:09:47.233466 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:47.234343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:47.236967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1195/2379] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:09:47 2022] DELETE /identity/v3/projects/eb758cbe277847529945b2d633c6f860 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:47.241430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:47.243773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:47.305691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b0d28-f052-4293-92c5-da88b21acfe2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:47.321200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d5abf-9d03-4f10-ada1-d82d4aebf936 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:47.326381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:47.327258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:47.327539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:47.327883 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] PATH_INFO: `/v3/users/bda58cf35b384cc38b2d83e1a02b43f0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:47.329072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda58cf35b384cc38b2d83e1a02b43f0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:47.330995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:47.376324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:47.396407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda58cf35b384cc38b2d83e1a02b43f0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:47.398688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda58cf35b384cc38b2d83e1a02b43f0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:47.401038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda58cf35b384cc38b2d83e1a02b43f0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:47.409027 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b80ca6-2a2d-4355-a49e-3c21478c0059 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:47.410826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b80ca6-2a2d-4355-a49e-3c21478c0059 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:47.413230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b80ca6-2a2d-4355-a49e-3c21478c0059 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:47.413921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b80ca6-2a2d-4355-a49e-3c21478c0059 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:47.422327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-32b80ca6-2a2d-4355-a49e-3c21478c0059 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:47.436099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda58cf35b384cc38b2d83e1a02b43f0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:47.449266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda58cf35b384cc38b2d83e1a02b43f0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:47.466277 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1196/2380] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:09:47 2022] DELETE /identity/v3/users/bda58cf35b384cc38b2d83e1a02b43f0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:09:47.546317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b80ca6-2a2d-4355-a49e-3c21478c0059 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:47.549040 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b80ca6-2a2d-4355-a49e-3c21478c0059 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:47.575231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1185/2381] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:09:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:09:47.766812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:47.768560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:47.936517 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9960c56-d2c6-4ed6-b302-f648e7f5cf29 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:47.948058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cac7f-f062-43d0-a4d6-e24367086a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:47.949416 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cac7f-f062-43d0-a4d6-e24367086a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:47.949901 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cac7f-f062-43d0-a4d6-e24367086a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:47.957215 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cac7f-f062-43d0-a4d6-e24367086a29 admin admin] PATH_INFO: `/v3/projects/f29d3dc61c23486ba2aa70d92a724796` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:47.963820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cac7f-f062-43d0-a4d6-e24367086a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f29d3dc61c23486ba2aa70d92a724796)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:47.965922 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cac7f-f062-43d0-a4d6-e24367086a29 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:48.051589 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-031cac7f-f062-43d0-a4d6-e24367086a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f29d3dc61c23486ba2aa70d92a724796', 'request_id': 'req-031cac7f-f062-43d0-a4d6-e24367086a29'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:48.058769 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-031cac7f-f062-43d0-a4d6-e24367086a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f29d3dc61c23486ba2aa70d92a724796', 'request_id': 'req-031cac7f-f062-43d0-a4d6-e24367086a29'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:48.065465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-031cac7f-f062-43d0-a4d6-e24367086a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f29d3dc61c23486ba2aa70d92a724796', 'request_id': 'req-031cac7f-f062-43d0-a4d6-e24367086a29'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:48.087327 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-031cac7f-f062-43d0-a4d6-e24367086a29 admin admin] The token cache is being invalidate because project f29d3dc61c23486ba2aa70d92a724796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:09:48.088334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-031cac7f-f062-43d0-a4d6-e24367086a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:48.090965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-031cac7f-f062-43d0-a4d6-e24367086a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:48.093119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1197/2382] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:09:47 2022] DELETE /identity/v3/projects/f29d3dc61c23486ba2aa70d92a724796 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:48.099257 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868a75e-df1b-4d2e-95d8-e3f7d0740397 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:48.099544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868a75e-df1b-4d2e-95d8-e3f7d0740397 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:48.100170 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868a75e-df1b-4d2e-95d8-e3f7d0740397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:48.133566 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6868a75e-df1b-4d2e-95d8-e3f7d0740397 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:48.137103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6868a75e-df1b-4d2e-95d8-e3f7d0740397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:48.158260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:48.277395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1186/2383] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:09:48.286619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dd307-331c-4042-81e9-edc03130d625 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:48.287722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dd307-331c-4042-81e9-edc03130d625 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:48.288156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dd307-331c-4042-81e9-edc03130d625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:48.344492 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-958dd307-331c-4042-81e9-edc03130d625 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:48.346192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-958dd307-331c-4042-81e9-edc03130d625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:48.359779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1198/2384] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:48.363850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868a75e-df1b-4d2e-95d8-e3f7d0740397 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:48.375316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868a75e-df1b-4d2e-95d8-e3f7d0740397 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:48.380637 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bae7f0-c94e-45c7-8eef-85b4ff82af9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:48.381449 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bae7f0-c94e-45c7-8eef-85b4ff82af9e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:48.381579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bae7f0-c94e-45c7-8eef-85b4ff82af9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:48.381934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bae7f0-c94e-45c7-8eef-85b4ff82af9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:48.382660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bae7f0-c94e-45c7-8eef-85b4ff82af9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:09:48.383078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bae7f0-c94e-45c7-8eef-85b4ff82af9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:48.384700 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bae7f0-c94e-45c7-8eef-85b4ff82af9e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:48.400903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1187/2385] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:09:48.406272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dd307-331c-4042-81e9-edc03130d625 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:48.418187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dd307-331c-4042-81e9-edc03130d625 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:48.423647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981e05a-75bd-4d99-8220-5428306210a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:48.424457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981e05a-75bd-4d99-8220-5428306210a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:48.424766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981e05a-75bd-4d99-8220-5428306210a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:48.425133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981e05a-75bd-4d99-8220-5428306210a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:48.425920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981e05a-75bd-4d99-8220-5428306210a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:48.428188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981e05a-75bd-4d99-8220-5428306210a9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:48.464892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1199/2386] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:09:48 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:48.470201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bae7f0-c94e-45c7-8eef-85b4ff82af9e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:48.489176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bae7f0-c94e-45c7-8eef-85b4ff82af9e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:48.604758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd95121-5dd3-4114-a8f8-9de799403dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:48.605482 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd95121-5dd3-4114-a8f8-9de799403dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:48.605883 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd95121-5dd3-4114-a8f8-9de799403dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:48.606197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd95121-5dd3-4114-a8f8-9de799403dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:48.606904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd95121-5dd3-4114-a8f8-9de799403dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:48.608871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd95121-5dd3-4114-a8f8-9de799403dd7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:48.668765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1188/2387] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:09:48 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:48.672545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981e05a-75bd-4d99-8220-5428306210a9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:48.684645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981e05a-75bd-4d99-8220-5428306210a9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:48.690715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb74a0e-2268-434f-9eab-97661de05fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:48.691611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb74a0e-2268-434f-9eab-97661de05fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:48.692007 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb74a0e-2268-434f-9eab-97661de05fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:48.692351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb74a0e-2268-434f-9eab-97661de05fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:48.693198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb74a0e-2268-434f-9eab-97661de05fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:48.694902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb74a0e-2268-434f-9eab-97661de05fa1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:48.701501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:48.713403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1200/2388] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:09:48.717202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd95121-5dd3-4114-a8f8-9de799403dd7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:48.733639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd95121-5dd3-4114-a8f8-9de799403dd7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:48.739404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b1896-8b72-4fe4-9e32-20c8bb9a2ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:48.740318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1896-8b72-4fe4-9e32-20c8bb9a2ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:48.740785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1896-8b72-4fe4-9e32-20c8bb9a2ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:48.741218 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1896-8b72-4fe4-9e32-20c8bb9a2ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:48.742129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b1896-8b72-4fe4-9e32-20c8bb9a2ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:48.743926 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b1896-8b72-4fe4-9e32-20c8bb9a2ba0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:48.759357 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb74a0e-2268-434f-9eab-97661de05fa1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:48.765322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1189/2389] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:48.774370 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb74a0e-2268-434f-9eab-97661de05fa1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:48.784024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66400e7-310f-4fbd-9d91-5df415b96025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:48.785770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66400e7-310f-4fbd-9d91-5df415b96025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:48.786349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66400e7-310f-4fbd-9d91-5df415b96025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:48.787114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66400e7-310f-4fbd-9d91-5df415b96025 admin admin] PATH_INFO: `/v3/projects/d5481290a1154832804693352103ce54/users/6f85fc5053354c509443936173f884dc/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:48.816389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66400e7-310f-4fbd-9d91-5df415b96025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5481290a1154832804693352103ce54, user_id=6f85fc5053354c509443936173f884dc, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:48.821538 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66400e7-310f-4fbd-9d91-5df415b96025 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:48.867489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82063dd8-ed3d-4398-9a4e-f68a764024fd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:48.868052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82063dd8-ed3d-4398-9a4e-f68a764024fd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:48.868523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82063dd8-ed3d-4398-9a4e-f68a764024fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:48.870110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1201/2390] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:48 2022] PUT /identity/v3/projects/d5481290a1154832804693352103ce54/users/6f85fc5053354c509443936173f884dc/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:09:48.930978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-82063dd8-ed3d-4398-9a4e-f68a764024fd None None] MFA Rules not processed for user `6f85fc5053354c509443936173f884dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:48.933533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-82063dd8-ed3d-4398-9a4e-f68a764024fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:49.063745 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1190/2391] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:49.065259 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66400e7-310f-4fbd-9d91-5df415b96025 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:49.182305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66400e7-310f-4fbd-9d91-5df415b96025 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:49.189038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46cfd1-9ba1-485a-9f75-f3ccf7327db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:49.189856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46cfd1-9ba1-485a-9f75-f3ccf7327db2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:49.190421 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46cfd1-9ba1-485a-9f75-f3ccf7327db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:49.190839 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46cfd1-9ba1-485a-9f75-f3ccf7327db2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:49.191800 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46cfd1-9ba1-485a-9f75-f3ccf7327db2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:49.194526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46cfd1-9ba1-485a-9f75-f3ccf7327db2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:49.236066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1202/2392] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:09:49 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:49.240064 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82063dd8-ed3d-4398-9a4e-f68a764024fd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:49.275492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82063dd8-ed3d-4398-9a4e-f68a764024fd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:49.285749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:49.378839 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46cfd1-9ba1-485a-9f75-f3ccf7327db2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:49.382984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0b46cfd1-9ba1-485a-9f75-f3ccf7327db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:49.392708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143502e9-602c-4cdf-a55e-759fd50e028c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:49.393408 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143502e9-602c-4cdf-a55e-759fd50e028c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:49.393774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143502e9-602c-4cdf-a55e-759fd50e028c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:49.393954 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143502e9-602c-4cdf-a55e-759fd50e028c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:49.394738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143502e9-602c-4cdf-a55e-759fd50e028c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:49.396775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143502e9-602c-4cdf-a55e-759fd50e028c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:49.410827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46cfd1-9ba1-485a-9f75-f3ccf7327db2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:49.424611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daed00-4efe-4404-99df-fea26334b4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:49.425974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daed00-4efe-4404-99df-fea26334b4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:49.426335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daed00-4efe-4404-99df-fea26334b4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:49.426682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daed00-4efe-4404-99df-fea26334b4e1 admin admin] PATH_INFO: `/v3/users/61cb43c95c99413c852ef62ea41de950` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:49.427934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1191/2393] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:09:49 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:49.431345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143502e9-602c-4cdf-a55e-759fd50e028c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:49.431779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4daed00-4efe-4404-99df-fea26334b4e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61cb43c95c99413c852ef62ea41de950)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:49.434004 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4daed00-4efe-4404-99df-fea26334b4e1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:49.444364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143502e9-602c-4cdf-a55e-759fd50e028c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:49.452349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a63ebc-90bd-4d3c-bae8-62399645ea1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:49.453205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a63ebc-90bd-4d3c-bae8-62399645ea1f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:49.453602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a63ebc-90bd-4d3c-bae8-62399645ea1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:49.453986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a63ebc-90bd-4d3c-bae8-62399645ea1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:49.454842 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a63ebc-90bd-4d3c-bae8-62399645ea1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:49.457933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a63ebc-90bd-4d3c-bae8-62399645ea1f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:49.483738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1192/2394] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:09:49.487872 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a63ebc-90bd-4d3c-bae8-62399645ea1f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:49.521750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a63ebc-90bd-4d3c-bae8-62399645ea1f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:49.535082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c4daed00-4efe-4404-99df-fea26334b4e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61cb43c95c99413c852ef62ea41de950'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:49.537680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645bebc-cba9-4c69-aa04-5d3f1d968898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:49.538123 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c4daed00-4efe-4404-99df-fea26334b4e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61cb43c95c99413c852ef62ea41de950'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:49.539320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c4daed00-4efe-4404-99df-fea26334b4e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61cb43c95c99413c852ef62ea41de950'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:49.540749 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645bebc-cba9-4c69-aa04-5d3f1d968898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:49.542302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645bebc-cba9-4c69-aa04-5d3f1d968898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:49.543307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645bebc-cba9-4c69-aa04-5d3f1d968898 admin admin] PATH_INFO: `/v3/projects/5b024b3efc964efc87ea92386fe07ee4/users/4eb798ffe04740dbad36bad4f361ca6c/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:49.582018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645bebc-cba9-4c69-aa04-5d3f1d968898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b024b3efc964efc87ea92386fe07ee4, user_id=4eb798ffe04740dbad36bad4f361ca6c, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:49.586831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645bebc-cba9-4c69-aa04-5d3f1d968898 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:49.601149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c4daed00-4efe-4404-99df-fea26334b4e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61cb43c95c99413c852ef62ea41de950'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:49.608431 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1193/2395] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:49 2022] PUT /identity/v3/projects/5b024b3efc964efc87ea92386fe07ee4/users/4eb798ffe04740dbad36bad4f361ca6c/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:09:49.612294 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645bebc-cba9-4c69-aa04-5d3f1d968898 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:49.612922 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c4daed00-4efe-4404-99df-fea26334b4e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61cb43c95c99413c852ef62ea41de950'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:49.614208 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6645bebc-cba9-4c69-aa04-5d3f1d968898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:49.647193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1203/2396] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:09:49 2022] DELETE /identity/v3/users/61cb43c95c99413c852ef62ea41de950 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:49.652812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2f14-cb2c-4f21-9e03-16b9d24b950a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:49.653342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2f14-cb2c-4f21-9e03-16b9d24b950a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:49.653625 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2f14-cb2c-4f21-9e03-16b9d24b950a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:49.699846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-013a2f14-cb2c-4f21-9e03-16b9d24b950a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:49.701436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-013a2f14-cb2c-4f21-9e03-16b9d24b950a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:49.738168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645bebc-cba9-4c69-aa04-5d3f1d968898 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:49.742316 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:49.790318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1204/2397] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:49.812463 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a2f14-cb2c-4f21-9e03-16b9d24b950a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:49.821523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-013a2f14-cb2c-4f21-9e03-16b9d24b950a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:49.868733 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff39e3-9091-42d9-95ec-adb6cf2920fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:49.869510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff39e3-9091-42d9-95ec-adb6cf2920fc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:49.869984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff39e3-9091-42d9-95ec-adb6cf2920fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:49.870320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff39e3-9091-42d9-95ec-adb6cf2920fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:49.871085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff39e3-9091-42d9-95ec-adb6cf2920fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:49.872745 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff39e3-9091-42d9-95ec-adb6cf2920fc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:49.885794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1194/2398] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:49.889443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff39e3-9091-42d9-95ec-adb6cf2920fc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:49.891225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a2f14-cb2c-4f21-9e03-16b9d24b950a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:49.894994 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88f839-dec2-44a5-a87c-3868a6791943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:49.895641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88f839-dec2-44a5-a87c-3868a6791943 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:49.895955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88f839-dec2-44a5-a87c-3868a6791943 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:49.896374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88f839-dec2-44a5-a87c-3868a6791943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:49.907988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88f839-dec2-44a5-a87c-3868a6791943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:49.908359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff39e3-9091-42d9-95ec-adb6cf2920fc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:49.909513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88f839-dec2-44a5-a87c-3868a6791943 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:49.920211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e5b69-fd16-4f2a-9cf5-fb456264c163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:49.920878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1205/2399] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:09:49.921361 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e5b69-fd16-4f2a-9cf5-fb456264c163 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:49.921700 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e5b69-fd16-4f2a-9cf5-fb456264c163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:49.922053 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e5b69-fd16-4f2a-9cf5-fb456264c163 admin admin] PATH_INFO: `/v3/projects/5b024b3efc964efc87ea92386fe07ee4/users/4eb798ffe04740dbad36bad4f361ca6c/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:49.925029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e5b69-fd16-4f2a-9cf5-fb456264c163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b024b3efc964efc87ea92386fe07ee4, user_id=4eb798ffe04740dbad36bad4f361ca6c, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:49.932219 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e5b69-fd16-4f2a-9cf5-fb456264c163 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:49.939588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:49.965360 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1195/2400] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:49 2022] PUT /identity/v3/projects/5b024b3efc964efc87ea92386fe07ee4/users/4eb798ffe04740dbad36bad4f361ca6c/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:09:49.977475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eed00e-3440-46f5-b5af-b28021246c73 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:49.977718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eed00e-3440-46f5-b5af-b28021246c73 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:49.978018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eed00e-3440-46f5-b5af-b28021246c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:50.014315 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a7eed00e-3440-46f5-b5af-b28021246c73 None None] MFA Rules not processed for user `4eb798ffe04740dbad36bad4f361ca6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:50.016767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a7eed00e-3440-46f5-b5af-b28021246c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:50.113842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1206/2401] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:50.126797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e5b69-fd16-4f2a-9cf5-fb456264c163 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:50.129761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0b9e5b69-fd16-4f2a-9cf5-fb456264c163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:50.243927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e5b69-fd16-4f2a-9cf5-fb456264c163 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:50.262724 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4594595-dc81-49bd-ad72-e49e5298f6df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:50.263560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4594595-dc81-49bd-ad72-e49e5298f6df service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:50.263724 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4594595-dc81-49bd-ad72-e49e5298f6df service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:50.264535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4594595-dc81-49bd-ad72-e49e5298f6df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:50.288239 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4594595-dc81-49bd-ad72-e49e5298f6df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:50.289999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4594595-dc81-49bd-ad72-e49e5298f6df service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:50.316758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1196/2402] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:09:50.505237 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eed00e-3440-46f5-b5af-b28021246c73 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:50.507425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a7eed00e-3440-46f5-b5af-b28021246c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:50.635547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eed00e-3440-46f5-b5af-b28021246c73 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:50.653936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:50.655066 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:50.655669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:50.656062 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] PATH_INFO: `/v3/projects/5602b9db0a89433894a93fe39df5f51e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:50.658821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5602b9db0a89433894a93fe39df5f51e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:50.663076 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:50.669311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:50.754271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5602b9db0a89433894a93fe39df5f51e', 'request_id': 'req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:50.770451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5602b9db0a89433894a93fe39df5f51e', 'request_id': 'req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:50.786752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5602b9db0a89433894a93fe39df5f51e', 'request_id': 'req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:50.812156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] The token cache is being invalidate because project 5602b9db0a89433894a93fe39df5f51e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:09:50.812660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:50.816578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:50.821379 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1207/2403] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:09:50 2022] DELETE /identity/v3/projects/5602b9db0a89433894a93fe39df5f51e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:50.828580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8487339-6294-4c95-b539-82b1d1b8e042 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:50.829515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8487339-6294-4c95-b539-82b1d1b8e042 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:50.830001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8487339-6294-4c95-b539-82b1d1b8e042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:50.872931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c8487339-6294-4c95-b539-82b1d1b8e042 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:50.874512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c8487339-6294-4c95-b539-82b1d1b8e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:50.933088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:50.934623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:51.000180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1197/2404] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:51.007645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487f23c-a2f0-49d8-94e7-141a27474ae8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:51.008825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487f23c-a2f0-49d8-94e7-141a27474ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:51.011285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487f23c-a2f0-49d8-94e7-141a27474ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:51.012019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd0ea5-210f-4b50-a990-2fe3bdaff08e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:51.017222 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f502e4-78d0-4e76-bb73-1799d0a99fcc tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb798ffe04740dbad36bad4f361ca6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b024b3efc964efc87ea92386fe07ee4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:51.018229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f502e4-78d0-4e76-bb73-1799d0a99fcc tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:51.018537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f502e4-78d0-4e76-bb73-1799d0a99fcc tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:51.018824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f502e4-78d0-4e76-bb73-1799d0a99fcc tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:51.019563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f502e4-78d0-4e76-bb73-1799d0a99fcc tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:09:51.019925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f502e4-78d0-4e76-bb73-1799d0a99fcc tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:51.021735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f502e4-78d0-4e76-bb73-1799d0a99fcc tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:51.039021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1208/2405] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:09:51.039956 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f502e4-78d0-4e76-bb73-1799d0a99fcc tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:51.050829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f502e4-78d0-4e76-bb73-1799d0a99fcc tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:51.055646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9160d4d-12c6-4d99-a3f2-96349f858578 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb798ffe04740dbad36bad4f361ca6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b024b3efc964efc87ea92386fe07ee4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:51.056356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9160d4d-12c6-4d99-a3f2-96349f858578 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:51.057175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9160d4d-12c6-4d99-a3f2-96349f858578 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:51.057458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9160d4d-12c6-4d99-a3f2-96349f858578 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:51.058338 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9160d4d-12c6-4d99-a3f2-96349f858578 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:51.060611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9160d4d-12c6-4d99-a3f2-96349f858578 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:51.064038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b487f23c-a2f0-49d8-94e7-141a27474ae8 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:51.074210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b487f23c-a2f0-49d8-94e7-141a27474ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:51.086776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1198/2406] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:51.089099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487f23c-a2f0-49d8-94e7-141a27474ae8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:51.107131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1209/2407] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:09:51 2022] POST /identity/v3/projects => generated 355 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:51.112348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9160d4d-12c6-4d99-a3f2-96349f858578 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:51.125302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487f23c-a2f0-49d8-94e7-141a27474ae8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:51.216902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9160d4d-12c6-4d99-a3f2-96349f858578 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:51.223079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef534-4b55-4502-8813-0e2f034fdd83 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb798ffe04740dbad36bad4f361ca6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b024b3efc964efc87ea92386fe07ee4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:51.223791 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef534-4b55-4502-8813-0e2f034fdd83 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:51.224073 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef534-4b55-4502-8813-0e2f034fdd83 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:51.224353 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef534-4b55-4502-8813-0e2f034fdd83 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:51.225091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef534-4b55-4502-8813-0e2f034fdd83 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:09:51.226456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef534-4b55-4502-8813-0e2f034fdd83 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:51.227022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef534-4b55-4502-8813-0e2f034fdd83 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:51.251932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:51.276103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1210/2408] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:09:51.276475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d8ad8-bf08-45ab-a707-d1251c5e32a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:51.276475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d8ad8-bf08-45ab-a707-d1251c5e32a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:51.276475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d8ad8-bf08-45ab-a707-d1251c5e32a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:51.276475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d8ad8-bf08-45ab-a707-d1251c5e32a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:51.276826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d8ad8-bf08-45ab-a707-d1251c5e32a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:09:51.276826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d8ad8-bf08-45ab-a707-d1251c5e32a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:51.278475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d8ad8-bf08-45ab-a707-d1251c5e32a4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:51.292620 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1199/2409] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:09:51.296967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef534-4b55-4502-8813-0e2f034fdd83 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:51.311053 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef534-4b55-4502-8813-0e2f034fdd83 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:51.318144 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da31707-59d2-479e-b14b-5d794b39a8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:51.318821 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da31707-59d2-479e-b14b-5d794b39a8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:51.319097 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da31707-59d2-479e-b14b-5d794b39a8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:51.319497 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da31707-59d2-479e-b14b-5d794b39a8b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:51.320234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da31707-59d2-479e-b14b-5d794b39a8b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:51.323136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da31707-59d2-479e-b14b-5d794b39a8b0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:51.331216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d8ad8-bf08-45ab-a707-d1251c5e32a4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:51.345261 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d8ad8-bf08-45ab-a707-d1251c5e32a4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:51.364598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb798ffe04740dbad36bad4f361ca6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b024b3efc964efc87ea92386fe07ee4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:51.367095 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:51.367486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:51.367898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] PATH_INFO: `/v3/projects/97388b9f4e684ea4844381e6133859c3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:51.370870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: Authorizing `identity:delete_project(project_id=97388b9f4e684ea4844381e6133859c3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:51.374757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1211/2410] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:09:51 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:51.379574 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:51.379883 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da31707-59d2-479e-b14b-5d794b39a8b0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:51.419619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:51.459633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97388b9f4e684ea4844381e6133859c3', 'request_id': 'req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:51.471404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97388b9f4e684ea4844381e6133859c3', 'request_id': 'req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:51.473433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da31707-59d2-479e-b14b-5d794b39a8b0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:51.488110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97388b9f4e684ea4844381e6133859c3', 'request_id': 'req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:51.507950 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] The token cache is being invalidate because project 97388b9f4e684ea4844381e6133859c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:09:51.508333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:51.510603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:51.513807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1200/2411] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:09:51 2022] DELETE /identity/v3/projects/97388b9f4e684ea4844381e6133859c3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:09:51.563195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07a1d0-b4b4-4573-980d-b75efc0f12f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:51.564094 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07a1d0-b4b4-4573-980d-b75efc0f12f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:51.564385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07a1d0-b4b4-4573-980d-b75efc0f12f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:51.565084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07a1d0-b4b4-4573-980d-b75efc0f12f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:51.566292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07a1d0-b4b4-4573-980d-b75efc0f12f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:51.570319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07a1d0-b4b4-4573-980d-b75efc0f12f6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:51.610279 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1212/2412] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:09:51 2022] POST /identity/v3/users => generated 287 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:51.610971 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:51.612844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:51.628401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8761e8c-adb8-48d6-9197-2ffaf6c7ade0 tempest-VolumeQuotaClassesTest-746018994 tempest-VolumeQuotaClassesTest-746018994-project-admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:51.648990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe9a7f-a180-4872-a510-fcf0303e242b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:51.650285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe9a7f-a180-4872-a510-fcf0303e242b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:51.651532 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe9a7f-a180-4872-a510-fcf0303e242b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:51.652117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe9a7f-a180-4872-a510-fcf0303e242b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:51.656885 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe9a7f-a180-4872-a510-fcf0303e242b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:51.658864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe9a7f-a180-4872-a510-fcf0303e242b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:51.688271 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1201/2413] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:09:51.692655 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07a1d0-b4b4-4573-980d-b75efc0f12f6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:51.703975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07a1d0-b4b4-4573-980d-b75efc0f12f6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:51.709425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b814dd5-2699-4d05-b316-08c424aa2542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:51.710150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b814dd5-2699-4d05-b316-08c424aa2542 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:51.710433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b814dd5-2699-4d05-b316-08c424aa2542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:51.710703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b814dd5-2699-4d05-b316-08c424aa2542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:51.711516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b814dd5-2699-4d05-b316-08c424aa2542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:51.714037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b814dd5-2699-4d05-b316-08c424aa2542 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:51.740916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1213/2414] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:51.741855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe9a7f-a180-4872-a510-fcf0303e242b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:51.753266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe9a7f-a180-4872-a510-fcf0303e242b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:51.765757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a87ffd-6a52-4ea4-b0fa-6575bf92bd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:51.765757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a87ffd-6a52-4ea4-b0fa-6575bf92bd2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:51.765757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a87ffd-6a52-4ea4-b0fa-6575bf92bd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:51.766405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a87ffd-6a52-4ea4-b0fa-6575bf92bd2c admin admin] PATH_INFO: `/v3/projects/022a79ab105348b2bdb0ea9a33a7db5d/users/58193a8749434527a019fe2f8d92ee90/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:51.785589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a87ffd-6a52-4ea4-b0fa-6575bf92bd2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=022a79ab105348b2bdb0ea9a33a7db5d, user_id=58193a8749434527a019fe2f8d92ee90, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:51.788657 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a87ffd-6a52-4ea4-b0fa-6575bf92bd2c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:51.812009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1202/2415] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:51 2022] PUT /identity/v3/projects/022a79ab105348b2bdb0ea9a33a7db5d/users/58193a8749434527a019fe2f8d92ee90/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:09:51.822950 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9ac4f-cf90-490d-a571-8ea07cc65f60 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:51.823239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9ac4f-cf90-490d-a571-8ea07cc65f60 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:51.823515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9ac4f-cf90-490d-a571-8ea07cc65f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:51.864732 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-52d9ac4f-cf90-490d-a571-8ea07cc65f60 None None] MFA Rules not processed for user `58193a8749434527a019fe2f8d92ee90`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:51.866179 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-52d9ac4f-cf90-490d-a571-8ea07cc65f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:51.883718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a87ffd-6a52-4ea4-b0fa-6575bf92bd2c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:51.886799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-75a87ffd-6a52-4ea4-b0fa-6575bf92bd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:51.941271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:51.968160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1214/2416] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:51.980453 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9ac4f-cf90-490d-a571-8ea07cc65f60 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:51.982508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-52d9ac4f-cf90-490d-a571-8ea07cc65f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:52.077906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a87ffd-6a52-4ea4-b0fa-6575bf92bd2c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:52.082680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9ac4f-cf90-490d-a571-8ea07cc65f60 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:52.087961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f15edf8-fa2e-4ac7-b11d-b2a5e5f1d2df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:52.089114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15edf8-fa2e-4ac7-b11d-b2a5e5f1d2df service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:52.089495 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15edf8-fa2e-4ac7-b11d-b2a5e5f1d2df service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:52.089931 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15edf8-fa2e-4ac7-b11d-b2a5e5f1d2df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:52.091914 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:52.093097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:52.093543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:52.094004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] PATH_INFO: `/v3/users/6f85fc5053354c509443936173f884dc` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:52.097666 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f85fc5053354c509443936173f884dc)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:52.100152 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:52.101829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f15edf8-fa2e-4ac7-b11d-b2a5e5f1d2df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:52.105475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f15edf8-fa2e-4ac7-b11d-b2a5e5f1d2df service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:52.128454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1215/2417] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:09:52.176608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f85fc5053354c509443936173f884dc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:52.177674 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f85fc5053354c509443936173f884dc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:52.193680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f85fc5053354c509443936173f884dc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:52.213269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f85fc5053354c509443936173f884dc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:52.220837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:52.227120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f85fc5053354c509443936173f884dc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:52.239822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1203/2418] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:09:51 2022] DELETE /identity/v3/users/6f85fc5053354c509443936173f884dc => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:52.245038 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea599494-91e5-437b-9cbd-16669fbc9638 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:52.245541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea599494-91e5-437b-9cbd-16669fbc9638 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:52.245776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea599494-91e5-437b-9cbd-16669fbc9638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:52.269952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ea599494-91e5-437b-9cbd-16669fbc9638 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:52.271696 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ea599494-91e5-437b-9cbd-16669fbc9638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:52.358251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1216/2419] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:52.367815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:52.370169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:52.538468 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5425180-469e-4b7c-b4b6-e9785fa1068e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:52.549532 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df4d03-e9e6-44ac-a79f-0baedffec58c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:52.550228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df4d03-e9e6-44ac-a79f-0baedffec58c service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:52.550507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df4d03-e9e6-44ac-a79f-0baedffec58c service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:52.550778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df4d03-e9e6-44ac-a79f-0baedffec58c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:52.565456 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df4d03-e9e6-44ac-a79f-0baedffec58c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:52.568770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df4d03-e9e6-44ac-a79f-0baedffec58c service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:52.605715 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1204/2420] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:09:53.146273 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea599494-91e5-437b-9cbd-16669fbc9638 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:53.148409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ea599494-91e5-437b-9cbd-16669fbc9638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:53.163039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea599494-91e5-437b-9cbd-16669fbc9638 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:53.169805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:53.170859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:53.171036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:53.171333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] PATH_INFO: `/v3/projects/d5481290a1154832804693352103ce54` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:53.172661 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5481290a1154832804693352103ce54)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:53.174761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:53.257927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5481290a1154832804693352103ce54', 'request_id': 'req-44ab538b-74c0-4220-8ec4-37b647884205'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:53.267117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5481290a1154832804693352103ce54', 'request_id': 'req-44ab538b-74c0-4220-8ec4-37b647884205'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:53.274454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5481290a1154832804693352103ce54', 'request_id': 'req-44ab538b-74c0-4220-8ec4-37b647884205'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:53.294721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] The token cache is being invalidate because project d5481290a1154832804693352103ce54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:09:53.295305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:53.296219 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:53.298667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1217/2421] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:09:53 2022] DELETE /identity/v3/projects/d5481290a1154832804693352103ce54 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:09:53.302451 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df4d03-e9e6-44ac-a79f-0baedffec58c service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:53.320846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df4d03-e9e6-44ac-a79f-0baedffec58c service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:53.424989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21867534-621d-4605-a211-33f680973560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:53.426605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21867534-621d-4605-a211-33f680973560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:53.427190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21867534-621d-4605-a211-33f680973560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:53.427675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21867534-621d-4605-a211-33f680973560 admin admin] PATH_INFO: `/v3/users/4eb798ffe04740dbad36bad4f361ca6c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:53.430133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21867534-621d-4605-a211-33f680973560 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eb798ffe04740dbad36bad4f361ca6c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:53.432347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21867534-621d-4605-a211-33f680973560 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:53.530520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-21867534-621d-4605-a211-33f680973560 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eb798ffe04740dbad36bad4f361ca6c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:53.532285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-21867534-621d-4605-a211-33f680973560 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eb798ffe04740dbad36bad4f361ca6c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:53.545311 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-21867534-621d-4605-a211-33f680973560 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eb798ffe04740dbad36bad4f361ca6c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:53.564999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-21867534-621d-4605-a211-33f680973560 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eb798ffe04740dbad36bad4f361ca6c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:53.577829 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-21867534-621d-4605-a211-33f680973560 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eb798ffe04740dbad36bad4f361ca6c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:53.588077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1205/2422] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:09:53 2022] DELETE /identity/v3/users/4eb798ffe04740dbad36bad4f361ca6c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:53.990795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:53.992649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:54.123929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab538b-74c0-4220-8ec4-37b647884205 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:54.181560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:54.318144 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:54.319254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:54.319641 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:54.320117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3 admin admin] PATH_INFO: `/v3/projects/5b024b3efc964efc87ea92386fe07ee4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:54.323205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b024b3efc964efc87ea92386fe07ee4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:54.327708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:54.356669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21867534-621d-4605-a211-33f680973560 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:54.358232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-21867534-621d-4605-a211-33f680973560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:54.402297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:54.414878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b024b3efc964efc87ea92386fe07ee4', 'request_id': 'req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:54.428308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b024b3efc964efc87ea92386fe07ee4', 'request_id': 'req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:54.437481 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b024b3efc964efc87ea92386fe07ee4', 'request_id': 'req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:54.460599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3 admin admin] The token cache is being invalidate because project 5b024b3efc964efc87ea92386fe07ee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:09:54.460995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:54.462269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a9eb682a-e725-4d87-8d9d-62ce7756b8b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:54.469906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1218/2423] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:09:53 2022] DELETE /identity/v3/projects/5b024b3efc964efc87ea92386fe07ee4 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:09:54.472367 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c3650-6cfe-413e-b609-7f9b979c9dc5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:54.472788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c3650-6cfe-413e-b609-7f9b979c9dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:54.473228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c3650-6cfe-413e-b609-7f9b979c9dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:54.508563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21867534-621d-4605-a211-33f680973560 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:54.522527 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-143c3650-6cfe-413e-b609-7f9b979c9dc5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:54.525585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-143c3650-6cfe-413e-b609-7f9b979c9dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:54.644750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32d5f1-7feb-494b-8a56-89ae1dfaea66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:54.645968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32d5f1-7feb-494b-8a56-89ae1dfaea66 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:54.646258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32d5f1-7feb-494b-8a56-89ae1dfaea66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:54.646536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32d5f1-7feb-494b-8a56-89ae1dfaea66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:54.648773 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1219/2424] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:54.658166 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-db32d5f1-7feb-494b-8a56-89ae1dfaea66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:54.660036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aa85e-3f8c-410b-a10c-69c8b3cc8cd9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:54.660527 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aa85e-3f8c-410b-a10c-69c8b3cc8cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:54.660984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aa85e-3f8c-410b-a10c-69c8b3cc8cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:54.700461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-257aa85e-3f8c-410b-a10c-69c8b3cc8cd9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:54.701942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-257aa85e-3f8c-410b-a10c-69c8b3cc8cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:54.710307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32d5f1-7feb-494b-8a56-89ae1dfaea66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:54.712826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32d5f1-7feb-494b-8a56-89ae1dfaea66 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:54.718843 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1220/2425] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:54.721859 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257aa85e-3f8c-410b-a10c-69c8b3cc8cd9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:54.743691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1206/2426] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:09:54.745917 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257aa85e-3f8c-410b-a10c-69c8b3cc8cd9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:54.761033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c22d1-b2f8-474b-912d-423fc3cec41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:54.762916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c22d1-b2f8-474b-912d-423fc3cec41e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:54.767376 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c22d1-b2f8-474b-912d-423fc3cec41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:54.767775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c22d1-b2f8-474b-912d-423fc3cec41e admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:54.768763 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c22d1-b2f8-474b-912d-423fc3cec41e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:09:54.769150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c22d1-b2f8-474b-912d-423fc3cec41e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:54.772316 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c22d1-b2f8-474b-912d-423fc3cec41e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:54.787608 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1221/2427] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:09:54.793201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32d5f1-7feb-494b-8a56-89ae1dfaea66 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:54.812329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32d5f1-7feb-494b-8a56-89ae1dfaea66 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:54.824304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fea2a-94ca-44a4-a2fb-ee503d2e12f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:54.826040 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fea2a-94ca-44a4-a2fb-ee503d2e12f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:54.826937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fea2a-94ca-44a4-a2fb-ee503d2e12f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:54.827984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fea2a-94ca-44a4-a2fb-ee503d2e12f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:54.829690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fea2a-94ca-44a4-a2fb-ee503d2e12f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:54.835366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fea2a-94ca-44a4-a2fb-ee503d2e12f2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:54.893766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c22d1-b2f8-474b-912d-423fc3cec41e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:54.902258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1207/2428] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:09:54 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:54.907461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c22d1-b2f8-474b-912d-423fc3cec41e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:54.962748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:55.034238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956d0e7-15e4-4772-9958-9f586f30ab92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:55.035030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956d0e7-15e4-4772-9958-9f586f30ab92 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:55.035584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956d0e7-15e4-4772-9958-9f586f30ab92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:55.035961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956d0e7-15e4-4772-9958-9f586f30ab92 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:55.037443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956d0e7-15e4-4772-9958-9f586f30ab92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:55.039938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956d0e7-15e4-4772-9958-9f586f30ab92 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:55.084871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fea2a-94ca-44a4-a2fb-ee503d2e12f2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:55.110681 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1222/2429] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:09:54 2022] POST /identity/v3/users => generated 296 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:55.124843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fea2a-94ca-44a4-a2fb-ee503d2e12f2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:55.141362 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd70ed3-8124-411d-b9e3-8c0f3b8e539c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:55.142309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd70ed3-8124-411d-b9e3-8c0f3b8e539c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:55.142805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd70ed3-8124-411d-b9e3-8c0f3b8e539c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:55.143272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd70ed3-8124-411d-b9e3-8c0f3b8e539c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:55.144512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd70ed3-8124-411d-b9e3-8c0f3b8e539c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:55.147881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd70ed3-8124-411d-b9e3-8c0f3b8e539c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:55.176450 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1208/2430] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:55.182212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956d0e7-15e4-4772-9958-9f586f30ab92 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:55.200288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956d0e7-15e4-4772-9958-9f586f30ab92 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:55.206372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1ede6-57b3-478d-981a-9e6a4348f331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:55.207253 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1ede6-57b3-478d-981a-9e6a4348f331 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:55.207621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1ede6-57b3-478d-981a-9e6a4348f331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:55.207985 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1ede6-57b3-478d-981a-9e6a4348f331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:55.208733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1ede6-57b3-478d-981a-9e6a4348f331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:55.210805 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1ede6-57b3-478d-981a-9e6a4348f331 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:55.232043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd70ed3-8124-411d-b9e3-8c0f3b8e539c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:55.240139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1223/2431] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:55.248295 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd70ed3-8124-411d-b9e3-8c0f3b8e539c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:55.255075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cad246-5ff2-42bb-a9a1-daa3dda6dc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:55.256189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cad246-5ff2-42bb-a9a1-daa3dda6dc31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:55.256681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cad246-5ff2-42bb-a9a1-daa3dda6dc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:55.257051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cad246-5ff2-42bb-a9a1-daa3dda6dc31 admin admin] PATH_INFO: `/v3/projects/d437b0684cb54b038a3d464015bfeaa6/users/582a0f487e194fde84263ae0f00837d7/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:55.281659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cad246-5ff2-42bb-a9a1-daa3dda6dc31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d437b0684cb54b038a3d464015bfeaa6, user_id=582a0f487e194fde84263ae0f00837d7, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:55.297603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cad246-5ff2-42bb-a9a1-daa3dda6dc31 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:55.320016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1209/2432] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:55 2022] PUT /identity/v3/projects/d437b0684cb54b038a3d464015bfeaa6/users/582a0f487e194fde84263ae0f00837d7/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:55.331141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019e9a7-cb4b-4e2d-93ad-69f1f19c64cd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:55.332088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019e9a7-cb4b-4e2d-93ad-69f1f19c64cd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:55.332088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019e9a7-cb4b-4e2d-93ad-69f1f19c64cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:55.371914 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9019e9a7-cb4b-4e2d-93ad-69f1f19c64cd None None] MFA Rules not processed for user `582a0f487e194fde84263ae0f00837d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:55.374417 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9019e9a7-cb4b-4e2d-93ad-69f1f19c64cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:55.455638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1224/2433] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:55.460148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cad246-5ff2-42bb-a9a1-daa3dda6dc31 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:55.524283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cad246-5ff2-42bb-a9a1-daa3dda6dc31 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:55.530543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e3cf3-42d5-40f1-ba68-4152958eb7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:55.531389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e3cf3-42d5-40f1-ba68-4152958eb7e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:55.531617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e3cf3-42d5-40f1-ba68-4152958eb7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:55.532136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e3cf3-42d5-40f1-ba68-4152958eb7e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:55.533019 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e3cf3-42d5-40f1-ba68-4152958eb7e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:55.535300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e3cf3-42d5-40f1-ba68-4152958eb7e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:55.562767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1210/2434] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:09:55 2022] POST /identity/v3/projects => generated 393 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:55.566994 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019e9a7-cb4b-4e2d-93ad-69f1f19c64cd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:55.578472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019e9a7-cb4b-4e2d-93ad-69f1f19c64cd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:55.634737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f1667-7fc3-4d9f-b1a2-5bb0b31e44c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:55.636120 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f1667-7fc3-4d9f-b1a2-5bb0b31e44c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:55.636415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f1667-7fc3-4d9f-b1a2-5bb0b31e44c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:55.636726 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f1667-7fc3-4d9f-b1a2-5bb0b31e44c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:55.637431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f1667-7fc3-4d9f-b1a2-5bb0b31e44c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:55.639434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f1667-7fc3-4d9f-b1a2-5bb0b31e44c0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:55.662875 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1225/2435] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:09:55 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:55.664343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e3cf3-42d5-40f1-ba68-4152958eb7e0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:55.669947 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:55.676071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e3cf3-42d5-40f1-ba68-4152958eb7e0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:55.681157 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38470baf-54bf-4606-be3a-704885156cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:55.682443 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38470baf-54bf-4606-be3a-704885156cab admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:55.682790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38470baf-54bf-4606-be3a-704885156cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:55.683075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38470baf-54bf-4606-be3a-704885156cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:55.683841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38470baf-54bf-4606-be3a-704885156cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:55.685430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38470baf-54bf-4606-be3a-704885156cab admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:55.696160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1211/2436] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:55.701013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f1667-7fc3-4d9f-b1a2-5bb0b31e44c0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:55.711275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f1667-7fc3-4d9f-b1a2-5bb0b31e44c0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:55.716133 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059edbee-ef85-43b1-94e3-9a2890f8ae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:55.717080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059edbee-ef85-43b1-94e3-9a2890f8ae60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:55.717432 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059edbee-ef85-43b1-94e3-9a2890f8ae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:55.717802 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059edbee-ef85-43b1-94e3-9a2890f8ae60 admin admin] PATH_INFO: `/v3/projects/9a7500c91dc5483ba3f3b58305c022b1/users/d27de4aab85a42ab931859fd604f0fff/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:55.732578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059edbee-ef85-43b1-94e3-9a2890f8ae60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7500c91dc5483ba3f3b58305c022b1, user_id=d27de4aab85a42ab931859fd604f0fff, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:55.735285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059edbee-ef85-43b1-94e3-9a2890f8ae60 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:55.749365 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1226/2437] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:55 2022] PUT /identity/v3/projects/9a7500c91dc5483ba3f3b58305c022b1/users/d27de4aab85a42ab931859fd604f0fff/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:55.753261 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38470baf-54bf-4606-be3a-704885156cab admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:55.764880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38470baf-54bf-4606-be3a-704885156cab admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:55.829048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d668c9-7b87-43d1-8f7b-5019120e680e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:55.829766 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d668c9-7b87-43d1-8f7b-5019120e680e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:55.830052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d668c9-7b87-43d1-8f7b-5019120e680e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:55.830333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d668c9-7b87-43d1-8f7b-5019120e680e admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:55.831156 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d668c9-7b87-43d1-8f7b-5019120e680e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:55.832833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d668c9-7b87-43d1-8f7b-5019120e680e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:55.847695 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1212/2438] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:09:55.849082 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059edbee-ef85-43b1-94e3-9a2890f8ae60 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:55.856396 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:55.862683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059edbee-ef85-43b1-94e3-9a2890f8ae60 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:55.868558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b15455-8f6c-4ca9-989f-5608c885d373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:55.869788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b15455-8f6c-4ca9-989f-5608c885d373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:55.870290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b15455-8f6c-4ca9-989f-5608c885d373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:55.870622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b15455-8f6c-4ca9-989f-5608c885d373 admin admin] PATH_INFO: `/v3/projects/9a7500c91dc5483ba3f3b58305c022b1/users/d27de4aab85a42ab931859fd604f0fff/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:55.873169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b15455-8f6c-4ca9-989f-5608c885d373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7500c91dc5483ba3f3b58305c022b1, user_id=d27de4aab85a42ab931859fd604f0fff, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:55.875907 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b15455-8f6c-4ca9-989f-5608c885d373 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:55.889210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1227/2439] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:55 2022] PUT /identity/v3/projects/9a7500c91dc5483ba3f3b58305c022b1/users/d27de4aab85a42ab931859fd604f0fff/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:55.899276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c070c-7380-4c2b-975e-7e107a8e326c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:55.899573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c070c-7380-4c2b-975e-7e107a8e326c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:55.899877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c070c-7380-4c2b-975e-7e107a8e326c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:55.943160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-972c070c-7380-4c2b-975e-7e107a8e326c None None] MFA Rules not processed for user `d27de4aab85a42ab931859fd604f0fff`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:55.944844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-972c070c-7380-4c2b-975e-7e107a8e326c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:56.074830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1213/2440] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:56.085968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b15455-8f6c-4ca9-989f-5608c885d373 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:56.088674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-17b15455-8f6c-4ca9-989f-5608c885d373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:56.207323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b15455-8f6c-4ca9-989f-5608c885d373 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:56.212257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb01442-a0c9-4022-9b95-3acd78eba5f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:56.213861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb01442-a0c9-4022-9b95-3acd78eba5f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:56.214239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb01442-a0c9-4022-9b95-3acd78eba5f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:56.214612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb01442-a0c9-4022-9b95-3acd78eba5f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:56.227451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb01442-a0c9-4022-9b95-3acd78eba5f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:56.229842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb01442-a0c9-4022-9b95-3acd78eba5f8 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:56.243181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1228/2441] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:09:56.319811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c070c-7380-4c2b-975e-7e107a8e326c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:56.324534 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-972c070c-7380-4c2b-975e-7e107a8e326c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:56.354898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c070c-7380-4c2b-975e-7e107a8e326c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:56.359336 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05fe46-b7c7-43c4-a4c4-a152f5648700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:56.360369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05fe46-b7c7-43c4-a4c4-a152f5648700 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:56.360914 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05fe46-b7c7-43c4-a4c4-a152f5648700 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:56.361258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05fe46-b7c7-43c4-a4c4-a152f5648700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:56.423947 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05fe46-b7c7-43c4-a4c4-a152f5648700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:56.425765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05fe46-b7c7-43c4-a4c4-a152f5648700 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:56.436418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1214/2442] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:09:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:09:58.270576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb01442-a0c9-4022-9b95-3acd78eba5f8 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:58.272614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0cb01442-a0c9-4022-9b95-3acd78eba5f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:58.371347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb01442-a0c9-4022-9b95-3acd78eba5f8 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:58.381070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a01447-2bf0-4a93-bf6b-31079592a98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:58.381960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a01447-2bf0-4a93-bf6b-31079592a98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:58.382471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a01447-2bf0-4a93-bf6b-31079592a98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:58.382863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a01447-2bf0-4a93-bf6b-31079592a98e admin admin] PATH_INFO: `/v3/users/df7d511abd8544878e7d0097a1ab0319` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:58.384136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a01447-2bf0-4a93-bf6b-31079592a98e admin admin] RBAC: Authorizing `identity:delete_user(user_id=df7d511abd8544878e7d0097a1ab0319)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:58.386398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a01447-2bf0-4a93-bf6b-31079592a98e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:58.493150 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f5a01447-2bf0-4a93-bf6b-31079592a98e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df7d511abd8544878e7d0097a1ab0319'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:58.494768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f5a01447-2bf0-4a93-bf6b-31079592a98e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df7d511abd8544878e7d0097a1ab0319'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:58.496056 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f5a01447-2bf0-4a93-bf6b-31079592a98e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df7d511abd8544878e7d0097a1ab0319'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:58.526393 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f5a01447-2bf0-4a93-bf6b-31079592a98e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df7d511abd8544878e7d0097a1ab0319'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:58.547174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f5a01447-2bf0-4a93-bf6b-31079592a98e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df7d511abd8544878e7d0097a1ab0319'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:58.570209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1229/2443] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:09:58 2022] DELETE /identity/v3/users/df7d511abd8544878e7d0097a1ab0319 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:58.823083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05fe46-b7c7-43c4-a4c4-a152f5648700 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:58.837850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05fe46-b7c7-43c4-a4c4-a152f5648700 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:58.848290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:58.914983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c39c55-ea11-4343-b4cb-f721a99da4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:58.916246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c39c55-ea11-4343-b4cb-f721a99da4da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:58.916705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c39c55-ea11-4343-b4cb-f721a99da4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:58.917022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c39c55-ea11-4343-b4cb-f721a99da4da admin admin] PATH_INFO: `/v3/projects/578cf1b29d474080a716e77fe2e84e5b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:58.918682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c39c55-ea11-4343-b4cb-f721a99da4da admin admin] RBAC: Authorizing `identity:delete_project(project_id=578cf1b29d474080a716e77fe2e84e5b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:58.920861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c39c55-ea11-4343-b4cb-f721a99da4da admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:58.984587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-55c39c55-ea11-4343-b4cb-f721a99da4da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '578cf1b29d474080a716e77fe2e84e5b', 'request_id': 'req-55c39c55-ea11-4343-b4cb-f721a99da4da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:59.000741 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-55c39c55-ea11-4343-b4cb-f721a99da4da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '578cf1b29d474080a716e77fe2e84e5b', 'request_id': 'req-55c39c55-ea11-4343-b4cb-f721a99da4da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:59.008002 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-55c39c55-ea11-4343-b4cb-f721a99da4da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '578cf1b29d474080a716e77fe2e84e5b', 'request_id': 'req-55c39c55-ea11-4343-b4cb-f721a99da4da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:59.020472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-55c39c55-ea11-4343-b4cb-f721a99da4da admin admin] The token cache is being invalidate because project 578cf1b29d474080a716e77fe2e84e5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:09:59.020661 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-55c39c55-ea11-4343-b4cb-f721a99da4da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:59.021417 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-55c39c55-ea11-4343-b4cb-f721a99da4da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:09:59.022986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1215/2444] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:09:58 2022] DELETE /identity/v3/projects/578cf1b29d474080a716e77fe2e84e5b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:59.028598 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae83ac-ee82-4569-b9e5-9339b8bc23fa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.028887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae83ac-ee82-4569-b9e5-9339b8bc23fa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.029188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae83ac-ee82-4569-b9e5-9339b8bc23fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.047229 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-13ae83ac-ee82-4569-b9e5-9339b8bc23fa None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:59.048464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-13ae83ac-ee82-4569-b9e5-9339b8bc23fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:59.053719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:59.111581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1230/2445] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:59.119519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bc419-5c51-487d-a810-f2344fcc02d1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.119846 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bc419-5c51-487d-a810-f2344fcc02d1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.120125 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bc419-5c51-487d-a810-f2344fcc02d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.140804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-707bc419-5c51-487d-a810-f2344fcc02d1 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:59.142031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-707bc419-5c51-487d-a810-f2344fcc02d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:59.150149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1216/2446] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:59.155144 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae83ac-ee82-4569-b9e5-9339b8bc23fa None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.166023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae83ac-ee82-4569-b9e5-9339b8bc23fa None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:59.171356 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dca31-1b62-4801-b7a4-84133f83bd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:59.172065 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dca31-1b62-4801-b7a4-84133f83bd58 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.172353 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dca31-1b62-4801-b7a4-84133f83bd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.172642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dca31-1b62-4801-b7a4-84133f83bd58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.173373 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dca31-1b62-4801-b7a4-84133f83bd58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:09:59.173699 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dca31-1b62-4801-b7a4-84133f83bd58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:59.175299 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dca31-1b62-4801-b7a4-84133f83bd58 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:59.191408 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1231/2447] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:09:59.192901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bc419-5c51-487d-a810-f2344fcc02d1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.202928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bc419-5c51-487d-a810-f2344fcc02d1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:59.208043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b677e63-4641-47ad-be62-4226652712e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:59.208701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b677e63-4641-47ad-be62-4226652712e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.208988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b677e63-4641-47ad-be62-4226652712e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.209265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b677e63-4641-47ad-be62-4226652712e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.209969 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b677e63-4641-47ad-be62-4226652712e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:59.211899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b677e63-4641-47ad-be62-4226652712e4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:59.240364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1217/2448] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:09:59 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:09:59.244516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dca31-1b62-4801-b7a4-84133f83bd58 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.255465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dca31-1b62-4801-b7a4-84133f83bd58 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:59.315310 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3fa46-b3e7-460d-b05e-3ed589b35810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:59.316023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3fa46-b3e7-460d-b05e-3ed589b35810 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.316313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3fa46-b3e7-460d-b05e-3ed589b35810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.316595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3fa46-b3e7-460d-b05e-3ed589b35810 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.317341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3fa46-b3e7-460d-b05e-3ed589b35810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:59.319313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3fa46-b3e7-460d-b05e-3ed589b35810 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:59.342841 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1232/2449] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:09:59 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:09:59.344396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b677e63-4641-47ad-be62-4226652712e4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.356199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b677e63-4641-47ad-be62-4226652712e4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:59.360897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a6a71-1873-4810-9d73-dd89cf084404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:59.361556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a6a71-1873-4810-9d73-dd89cf084404 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.361826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a6a71-1873-4810-9d73-dd89cf084404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.362099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a6a71-1873-4810-9d73-dd89cf084404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.362818 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a6a71-1873-4810-9d73-dd89cf084404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:59.364438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a6a71-1873-4810-9d73-dd89cf084404 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:59.374806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1218/2450] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:59.378760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3fa46-b3e7-460d-b05e-3ed589b35810 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.388815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3fa46-b3e7-460d-b05e-3ed589b35810 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:59.393559 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c0ec3-c1f2-4ae6-bd5f-288c6a78f90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:59.394208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c0ec3-c1f2-4ae6-bd5f-288c6a78f90e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.394490 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c0ec3-c1f2-4ae6-bd5f-288c6a78f90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.394764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c0ec3-c1f2-4ae6-bd5f-288c6a78f90e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.395485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c0ec3-c1f2-4ae6-bd5f-288c6a78f90e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:59.397125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c0ec3-c1f2-4ae6-bd5f-288c6a78f90e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:59.408383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1233/2451] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:59.413013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a6a71-1873-4810-9d73-dd89cf084404 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.427907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a6a71-1873-4810-9d73-dd89cf084404 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:59.432997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70f540-9565-4858-a169-9bd42312150b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:59.434032 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70f540-9565-4858-a169-9bd42312150b admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.434316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70f540-9565-4858-a169-9bd42312150b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.434593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70f540-9565-4858-a169-9bd42312150b admin admin] PATH_INFO: `/v3/projects/12936b8fa28f4bebb77f3689874d24ac/users/063702961afa471e81861e622a06b0f9/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.449382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70f540-9565-4858-a169-9bd42312150b admin admin] RBAC: Authorizing `identity:create_grant(project_id=12936b8fa28f4bebb77f3689874d24ac, user_id=063702961afa471e81861e622a06b0f9, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:59.452055 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70f540-9565-4858-a169-9bd42312150b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:59.468264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1219/2452] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:59 2022] PUT /identity/v3/projects/12936b8fa28f4bebb77f3689874d24ac/users/063702961afa471e81861e622a06b0f9/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:59.478358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79b937-b260-4af7-9a5c-abe847ed4b2e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.478721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79b937-b260-4af7-9a5c-abe847ed4b2e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.479608 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79b937-b260-4af7-9a5c-abe847ed4b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.508172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6e79b937-b260-4af7-9a5c-abe847ed4b2e None None] MFA Rules not processed for user `063702961afa471e81861e622a06b0f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:59.509413 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6e79b937-b260-4af7-9a5c-abe847ed4b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:59.573438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1234/2453] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:09:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:09:59.578033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70f540-9565-4858-a169-9bd42312150b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.630273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:59.650047 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70f540-9565-4858-a169-9bd42312150b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:59.654921 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ed389-2be8-431f-927e-912719116e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:59.655585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ed389-2be8-431f-927e-912719116e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.655916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ed389-2be8-431f-927e-912719116e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.656231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ed389-2be8-431f-927e-912719116e88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.657014 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ed389-2be8-431f-927e-912719116e88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:59.658934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ed389-2be8-431f-927e-912719116e88 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:59.697087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1220/2454] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:09:59 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:59.704633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79b937-b260-4af7-9a5c-abe847ed4b2e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.727863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79b937-b260-4af7-9a5c-abe847ed4b2e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:59.732350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:59.804402 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146046d-4350-414a-8b4c-99cf54910a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:59.805214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146046d-4350-414a-8b4c-99cf54910a82 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.805913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146046d-4350-414a-8b4c-99cf54910a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.805913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146046d-4350-414a-8b4c-99cf54910a82 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.806633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146046d-4350-414a-8b4c-99cf54910a82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:59.809380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146046d-4350-414a-8b4c-99cf54910a82 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:59.842062 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1235/2455] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:09:59 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:59.843462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ed389-2be8-431f-927e-912719116e88 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.861013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ed389-2be8-431f-927e-912719116e88 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:59.866488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9104c-0515-4e41-885f-e9b355092e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:59.867305 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9104c-0515-4e41-885f-e9b355092e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.868001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9104c-0515-4e41-885f-e9b355092e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.871113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9104c-0515-4e41-885f-e9b355092e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.872100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9104c-0515-4e41-885f-e9b355092e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:59.874023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9104c-0515-4e41-885f-e9b355092e93 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:59.885569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1221/2456] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:09:59.889816 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146046d-4350-414a-8b4c-99cf54910a82 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.900529 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146046d-4350-414a-8b4c-99cf54910a82 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:59.906592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a813c7-0f9c-4e9f-bf5b-fcb243a36e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:59.907621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a813c7-0f9c-4e9f-bf5b-fcb243a36e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.908096 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a813c7-0f9c-4e9f-bf5b-fcb243a36e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.908435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a813c7-0f9c-4e9f-bf5b-fcb243a36e5f admin admin] PATH_INFO: `/v3/projects/60bf899f0ce54b388fab8652f38a542f/users/4fc3af4b7eb24dcda2e713815cf33cea/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.932780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a813c7-0f9c-4e9f-bf5b-fcb243a36e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=60bf899f0ce54b388fab8652f38a542f, user_id=4fc3af4b7eb24dcda2e713815cf33cea, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:59.936020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a813c7-0f9c-4e9f-bf5b-fcb243a36e5f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:59.951611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1236/2457] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:09:59 2022] PUT /identity/v3/projects/60bf899f0ce54b388fab8652f38a542f/users/4fc3af4b7eb24dcda2e713815cf33cea/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:09:59.955721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9104c-0515-4e41-885f-e9b355092e93 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.967426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9104c-0515-4e41-885f-e9b355092e93 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.041429 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa87a61-7424-4205-b098-26ec63b2cff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.042320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa87a61-7424-4205-b098-26ec63b2cff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.042827 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa87a61-7424-4205-b098-26ec63b2cff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.043110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa87a61-7424-4205-b098-26ec63b2cff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.044469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa87a61-7424-4205-b098-26ec63b2cff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.046623 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa87a61-7424-4205-b098-26ec63b2cff9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.062413 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1222/2458] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:00.065872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a813c7-0f9c-4e9f-bf5b-fcb243a36e5f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.080591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a813c7-0f9c-4e9f-bf5b-fcb243a36e5f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.086488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bcec0-aadf-4df7-b1da-f27b6b978ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.088682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bcec0-aadf-4df7-b1da-f27b6b978ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.089045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bcec0-aadf-4df7-b1da-f27b6b978ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.089820 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bcec0-aadf-4df7-b1da-f27b6b978ac4 admin admin] PATH_INFO: `/v3/projects/60bf899f0ce54b388fab8652f38a542f/users/4fc3af4b7eb24dcda2e713815cf33cea/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.092117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bcec0-aadf-4df7-b1da-f27b6b978ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60bf899f0ce54b388fab8652f38a542f, user_id=4fc3af4b7eb24dcda2e713815cf33cea, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.095462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bcec0-aadf-4df7-b1da-f27b6b978ac4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.110359 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1237/2459] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:00 2022] PUT /identity/v3/projects/60bf899f0ce54b388fab8652f38a542f/users/4fc3af4b7eb24dcda2e713815cf33cea/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:00.124146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc61884-2d64-4276-953f-61d342294d96 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.124676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc61884-2d64-4276-953f-61d342294d96 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.125287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc61884-2d64-4276-953f-61d342294d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.163800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0cc61884-2d64-4276-953f-61d342294d96 None None] MFA Rules not processed for user `4fc3af4b7eb24dcda2e713815cf33cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.165171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0cc61884-2d64-4276-953f-61d342294d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.241441 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bcec0-aadf-4df7-b1da-f27b6b978ac4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.243372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9c6bcec0-aadf-4df7-b1da-f27b6b978ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.257750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1223/2460] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.270246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc61884-2d64-4276-953f-61d342294d96 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.271617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0cc61884-2d64-4276-953f-61d342294d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.307900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:00.317236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bcec0-aadf-4df7-b1da-f27b6b978ac4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.323051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.324179 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.324464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.324742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] PATH_INFO: `/v3/users/582a0f487e194fde84263ae0f00837d7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.325993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=582a0f487e194fde84263ae0f00837d7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.327999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.333692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc61884-2d64-4276-953f-61d342294d96 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.337391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef552f3-9c53-4fd3-8d54-914ce2cdac27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.338597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef552f3-9c53-4fd3-8d54-914ce2cdac27 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.338894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef552f3-9c53-4fd3-8d54-914ce2cdac27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.339172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef552f3-9c53-4fd3-8d54-914ce2cdac27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.348508 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef552f3-9c53-4fd3-8d54-914ce2cdac27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.349942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef552f3-9c53-4fd3-8d54-914ce2cdac27 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.361252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1224/2461] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:10:00.376181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '582a0f487e194fde84263ae0f00837d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:00.376902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '582a0f487e194fde84263ae0f00837d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:00.377546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '582a0f487e194fde84263ae0f00837d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:00.389346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:00.394738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '582a0f487e194fde84263ae0f00837d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:00.399578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '582a0f487e194fde84263ae0f00837d7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:00.415306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1238/2462] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:00 2022] DELETE /identity/v3/users/582a0f487e194fde84263ae0f00837d7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:00.420044 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97171a-27fb-441c-bf6a-e1be06004bcd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.420333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97171a-27fb-441c-bf6a-e1be06004bcd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.420608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97171a-27fb-441c-bf6a-e1be06004bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.437086 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.438563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.439721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bc97171a-27fb-441c-bf6a-e1be06004bcd None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.440916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bc97171a-27fb-441c-bf6a-e1be06004bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.498769 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc6a0d-0920-4a8e-b11c-84934ba2d7d2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.503159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30b4f4-d035-46b3-ac9e-eacb88fe147b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.503159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30b4f4-d035-46b3-ac9e-eacb88fe147b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.503759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30b4f4-d035-46b3-ac9e-eacb88fe147b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.503759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30b4f4-d035-46b3-ac9e-eacb88fe147b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.505891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7e30b4f4-d035-46b3-ac9e-eacb88fe147b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.509018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1225/2463] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.520713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97171a-27fb-441c-bf6a-e1be06004bcd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.522094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bc97171a-27fb-441c-bf6a-e1be06004bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.579786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30b4f4-d035-46b3-ac9e-eacb88fe147b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.581326 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30b4f4-d035-46b3-ac9e-eacb88fe147b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.583854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97171a-27fb-441c-bf6a-e1be06004bcd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.587736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee233ab-dc51-425e-b1f5-5533714e68be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.588413 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee233ab-dc51-425e-b1f5-5533714e68be service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.588805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee233ab-dc51-425e-b1f5-5533714e68be service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.589086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee233ab-dc51-425e-b1f5-5533714e68be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.590242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1239/2464] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:10:00.598512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee233ab-dc51-425e-b1f5-5533714e68be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.600096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee233ab-dc51-425e-b1f5-5533714e68be service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.610472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1226/2465] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:00.913592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30b4f4-d035-46b3-ac9e-eacb88fe147b service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.915899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7e30b4f4-d035-46b3-ac9e-eacb88fe147b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.930041 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30b4f4-d035-46b3-ac9e-eacb88fe147b service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.935100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b1626-a578-4652-8254-2414ca85dd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.936011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b1626-a578-4652-8254-2414ca85dd4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.936296 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b1626-a578-4652-8254-2414ca85dd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.936573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b1626-a578-4652-8254-2414ca85dd4f admin admin] PATH_INFO: `/v3/projects/d437b0684cb54b038a3d464015bfeaa6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.937763 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b1626-a578-4652-8254-2414ca85dd4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d437b0684cb54b038a3d464015bfeaa6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.939689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b1626-a578-4652-8254-2414ca85dd4f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.989669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a2b1626-a578-4652-8254-2414ca85dd4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd437b0684cb54b038a3d464015bfeaa6', 'request_id': 'req-5a2b1626-a578-4652-8254-2414ca85dd4f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:00.996751 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a2b1626-a578-4652-8254-2414ca85dd4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd437b0684cb54b038a3d464015bfeaa6', 'request_id': 'req-5a2b1626-a578-4652-8254-2414ca85dd4f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.003184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a2b1626-a578-4652-8254-2414ca85dd4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd437b0684cb54b038a3d464015bfeaa6', 'request_id': 'req-5a2b1626-a578-4652-8254-2414ca85dd4f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.016304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a2b1626-a578-4652-8254-2414ca85dd4f admin admin] The token cache is being invalidate because project d437b0684cb54b038a3d464015bfeaa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:01.016616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a2b1626-a578-4652-8254-2414ca85dd4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.017244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a2b1626-a578-4652-8254-2414ca85dd4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.019032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1240/2466] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:00 2022] DELETE /identity/v3/projects/d437b0684cb54b038a3d464015bfeaa6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:01.022764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee233ab-dc51-425e-b1f5-5533714e68be service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.032292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee233ab-dc51-425e-b1f5-5533714e68be service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.090526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.091357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.091641 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.091949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] PATH_INFO: `/v3/users/d27de4aab85a42ab931859fd604f0fff` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.093158 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d27de4aab85a42ab931859fd604f0fff)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.095122 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.117188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:01.145341 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd27de4aab85a42ab931859fd604f0fff'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.146033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd27de4aab85a42ab931859fd604f0fff'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.154225 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd27de4aab85a42ab931859fd604f0fff'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.166406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd27de4aab85a42ab931859fd604f0fff'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.173148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd27de4aab85a42ab931859fd604f0fff'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.178983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1227/2467] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:01 2022] DELETE /identity/v3/users/d27de4aab85a42ab931859fd604f0fff => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:01.283199 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13487ce2-2f55-4a2f-8ae0-0bd6474901a4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.283797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13487ce2-2f55-4a2f-8ae0-0bd6474901a4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.284331 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13487ce2-2f55-4a2f-8ae0-0bd6474901a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.308074 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-13487ce2-2f55-4a2f-8ae0-0bd6474901a4 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:01.309284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-13487ce2-2f55-4a2f-8ae0-0bd6474901a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:01.320301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:01.385415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1241/2468] 10.222.0.65 () {58 vars in 967 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:01.394016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.395725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:01.470337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb73846-10eb-4841-b066-5adfdca491fa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.475518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be261e8d-3274-4bb7-a54b-36e6edd97191 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.476206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be261e8d-3274-4bb7-a54b-36e6edd97191 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.476486 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be261e8d-3274-4bb7-a54b-36e6edd97191 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.476758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be261e8d-3274-4bb7-a54b-36e6edd97191 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.479733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13487ce2-2f55-4a2f-8ae0-0bd6474901a4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.481121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-13487ce2-2f55-4a2f-8ae0-0bd6474901a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:01.486601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be261e8d-3274-4bb7-a54b-36e6edd97191 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.488243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be261e8d-3274-4bb7-a54b-36e6edd97191 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.497474 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1228/2469] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:10:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:10:01.566950 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13487ce2-2f55-4a2f-8ae0-0bd6474901a4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.572065 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedbf4c-012a-473e-98b9-475ac4338139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.572971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedbf4c-012a-473e-98b9-475ac4338139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.573337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedbf4c-012a-473e-98b9-475ac4338139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.573567 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedbf4c-012a-473e-98b9-475ac4338139 admin admin] PATH_INFO: `/v3/projects/9a7500c91dc5483ba3f3b58305c022b1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.574851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedbf4c-012a-473e-98b9-475ac4338139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a7500c91dc5483ba3f3b58305c022b1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.576828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedbf4c-012a-473e-98b9-475ac4338139 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.628042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6fedbf4c-012a-473e-98b9-475ac4338139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a7500c91dc5483ba3f3b58305c022b1', 'request_id': 'req-6fedbf4c-012a-473e-98b9-475ac4338139'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.634375 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6fedbf4c-012a-473e-98b9-475ac4338139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a7500c91dc5483ba3f3b58305c022b1', 'request_id': 'req-6fedbf4c-012a-473e-98b9-475ac4338139'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.640983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6fedbf4c-012a-473e-98b9-475ac4338139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a7500c91dc5483ba3f3b58305c022b1', 'request_id': 'req-6fedbf4c-012a-473e-98b9-475ac4338139'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.657323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6fedbf4c-012a-473e-98b9-475ac4338139 admin admin] The token cache is being invalidate because project 9a7500c91dc5483ba3f3b58305c022b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:01.657575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6fedbf4c-012a-473e-98b9-475ac4338139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.658369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6fedbf4c-012a-473e-98b9-475ac4338139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:01.660437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1242/2470] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:01 2022] DELETE /identity/v3/projects/9a7500c91dc5483ba3f3b58305c022b1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:01.665356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b1f0c-933a-4592-9d55-f97bc4f521b5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.665600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b1f0c-933a-4592-9d55-f97bc4f521b5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.665935 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b1f0c-933a-4592-9d55-f97bc4f521b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.685796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-541b1f0c-933a-4592-9d55-f97bc4f521b5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:01.687260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-541b1f0c-933a-4592-9d55-f97bc4f521b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:01.754092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1229/2471] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:01.765346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbced9-a42a-41b8-b66d-be239b512b7a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.765984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbced9-a42a-41b8-b66d-be239b512b7a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.766469 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbced9-a42a-41b8-b66d-be239b512b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.794587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-84cbced9-a42a-41b8-b66d-be239b512b7a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:01.795989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-84cbced9-a42a-41b8-b66d-be239b512b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:01.805499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1243/2472] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:01.810259 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b1f0c-933a-4592-9d55-f97bc4f521b5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.820371 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b1f0c-933a-4592-9d55-f97bc4f521b5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.825956 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d0a49-571f-4958-b606-0ab09ca442eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.826318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d0a49-571f-4958-b606-0ab09ca442eb admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.826602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d0a49-571f-4958-b606-0ab09ca442eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.826883 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d0a49-571f-4958-b606-0ab09ca442eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.827616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d0a49-571f-4958-b606-0ab09ca442eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:01.827968 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d0a49-571f-4958-b606-0ab09ca442eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.829633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d0a49-571f-4958-b606-0ab09ca442eb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.842807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1230/2473] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:01.846779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbced9-a42a-41b8-b66d-be239b512b7a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.857390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbced9-a42a-41b8-b66d-be239b512b7a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.862266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309fac1-2c8a-4864-b0db-4041d2b95bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.862943 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309fac1-2c8a-4864-b0db-4041d2b95bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.863221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309fac1-2c8a-4864-b0db-4041d2b95bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.863509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309fac1-2c8a-4864-b0db-4041d2b95bc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.864249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309fac1-2c8a-4864-b0db-4041d2b95bc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.866468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309fac1-2c8a-4864-b0db-4041d2b95bc6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.899709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1244/2474] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:01.902729 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d0a49-571f-4958-b606-0ab09ca442eb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.913795 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d0a49-571f-4958-b606-0ab09ca442eb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.927450 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:01.979891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26385869-c216-41cf-b9c5-62b9ec0f2185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.980578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26385869-c216-41cf-b9c5-62b9ec0f2185 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.980892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26385869-c216-41cf-b9c5-62b9ec0f2185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.981173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26385869-c216-41cf-b9c5-62b9ec0f2185 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.981910 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26385869-c216-41cf-b9c5-62b9ec0f2185 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.983906 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26385869-c216-41cf-b9c5-62b9ec0f2185 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.009368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1231/2475] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:02.011147 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309fac1-2c8a-4864-b0db-4041d2b95bc6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.023613 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309fac1-2c8a-4864-b0db-4041d2b95bc6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.030861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a0a24-b19d-4b80-8be5-0769b5e78ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.031528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a0a24-b19d-4b80-8be5-0769b5e78ddf admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.031848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a0a24-b19d-4b80-8be5-0769b5e78ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.032129 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a0a24-b19d-4b80-8be5-0769b5e78ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.033493 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a0a24-b19d-4b80-8be5-0769b5e78ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.035109 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a0a24-b19d-4b80-8be5-0769b5e78ddf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.051112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1245/2476] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.052691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26385869-c216-41cf-b9c5-62b9ec0f2185 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.064436 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26385869-c216-41cf-b9c5-62b9ec0f2185 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.069790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae9aab-7dfb-4563-bacc-0336060f0119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.070760 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae9aab-7dfb-4563-bacc-0336060f0119 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.071244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae9aab-7dfb-4563-bacc-0336060f0119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.072060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae9aab-7dfb-4563-bacc-0336060f0119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.073679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae9aab-7dfb-4563-bacc-0336060f0119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.075511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae9aab-7dfb-4563-bacc-0336060f0119 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.091479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1232/2477] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.093813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a0a24-b19d-4b80-8be5-0769b5e78ddf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.105510 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a0a24-b19d-4b80-8be5-0769b5e78ddf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.118961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6734-bd6d-46b6-9ec5-d833f3943f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.120057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6734-bd6d-46b6-9ec5-d833f3943f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.121778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6734-bd6d-46b6-9ec5-d833f3943f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.121778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6734-bd6d-46b6-9ec5-d833f3943f4c admin admin] PATH_INFO: `/v3/projects/aabc22f4e29346bab822810573f531a7/users/90e32bab72924abfa9f4bf2053ead72f/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.130296 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:02.142080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f6734-bd6d-46b6-9ec5-d833f3943f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aabc22f4e29346bab822810573f531a7, user_id=90e32bab72924abfa9f4bf2053ead72f, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.145996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f6734-bd6d-46b6-9ec5-d833f3943f4c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.167872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1246/2478] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:02 2022] PUT /identity/v3/projects/aabc22f4e29346bab822810573f531a7/users/90e32bab72924abfa9f4bf2053ead72f/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:02.175368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03768611-6c3d-4a27-beb1-46f20ff2de95 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.175798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03768611-6c3d-4a27-beb1-46f20ff2de95 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.176089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03768611-6c3d-4a27-beb1-46f20ff2de95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.209253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-03768611-6c3d-4a27-beb1-46f20ff2de95 None None] MFA Rules not processed for user `90e32bab72924abfa9f4bf2053ead72f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:02.210730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-03768611-6c3d-4a27-beb1-46f20ff2de95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:02.294027 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1233/2479] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:02.295172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6734-bd6d-46b6-9ec5-d833f3943f4c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.357717 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6734-bd6d-46b6-9ec5-d833f3943f4c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.362236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e211923-740a-4076-b28c-07d1902c8668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.362884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e211923-740a-4076-b28c-07d1902c8668 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.363166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e211923-740a-4076-b28c-07d1902c8668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.363441 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e211923-740a-4076-b28c-07d1902c8668 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.364240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e211923-740a-4076-b28c-07d1902c8668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.366131 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e211923-740a-4076-b28c-07d1902c8668 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.390507 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1247/2480] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:02 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:02.394825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03768611-6c3d-4a27-beb1-46f20ff2de95 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.405890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03768611-6c3d-4a27-beb1-46f20ff2de95 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.463936 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375e1ab-0ef6-4217-abf7-c5bc96548835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.464605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375e1ab-0ef6-4217-abf7-c5bc96548835 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.464888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375e1ab-0ef6-4217-abf7-c5bc96548835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.465278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375e1ab-0ef6-4217-abf7-c5bc96548835 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.465973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375e1ab-0ef6-4217-abf7-c5bc96548835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.467901 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375e1ab-0ef6-4217-abf7-c5bc96548835 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.492183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1234/2481] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:02 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:10:02.493524 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e211923-740a-4076-b28c-07d1902c8668 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.505194 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e211923-740a-4076-b28c-07d1902c8668 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.510069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d5b35-0994-4e14-984b-f7aa8c755cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.510737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d5b35-0994-4e14-984b-f7aa8c755cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.511023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d5b35-0994-4e14-984b-f7aa8c755cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.511301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d5b35-0994-4e14-984b-f7aa8c755cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.512085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d5b35-0994-4e14-984b-f7aa8c755cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.513688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d5b35-0994-4e14-984b-f7aa8c755cc8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.525302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1248/2482] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.530310 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375e1ab-0ef6-4217-abf7-c5bc96548835 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.541507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375e1ab-0ef6-4217-abf7-c5bc96548835 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.546793 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6cb40-1e96-414d-b0b9-8f4c36e3ad58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.547628 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6cb40-1e96-414d-b0b9-8f4c36e3ad58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.547941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6cb40-1e96-414d-b0b9-8f4c36e3ad58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.548363 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6cb40-1e96-414d-b0b9-8f4c36e3ad58 admin admin] PATH_INFO: `/v3/projects/9920a2a7a53b4be5ba5fd70946f6f083/users/ff45383a8f2f467f9232d7d5380f4908/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.561955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:02.566463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6cb40-1e96-414d-b0b9-8f4c36e3ad58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9920a2a7a53b4be5ba5fd70946f6f083, user_id=ff45383a8f2f467f9232d7d5380f4908, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.569148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6cb40-1e96-414d-b0b9-8f4c36e3ad58 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.584173 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1235/2483] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:02 2022] PUT /identity/v3/projects/9920a2a7a53b4be5ba5fd70946f6f083/users/ff45383a8f2f467f9232d7d5380f4908/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:02.588222 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d5b35-0994-4e14-984b-f7aa8c755cc8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.598167 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d5b35-0994-4e14-984b-f7aa8c755cc8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.654969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac2541-c91a-44ff-8027-cebfbcfc20e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.655547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac2541-c91a-44ff-8027-cebfbcfc20e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.655864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac2541-c91a-44ff-8027-cebfbcfc20e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.656143 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac2541-c91a-44ff-8027-cebfbcfc20e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.656862 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac2541-c91a-44ff-8027-cebfbcfc20e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.658478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac2541-c91a-44ff-8027-cebfbcfc20e1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.668793 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1249/2484] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.673067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6cb40-1e96-414d-b0b9-8f4c36e3ad58 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.684006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6cb40-1e96-414d-b0b9-8f4c36e3ad58 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.688606 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ebcef-5db1-4ae5-8a45-72a199a34bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.689512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ebcef-5db1-4ae5-8a45-72a199a34bb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.689796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ebcef-5db1-4ae5-8a45-72a199a34bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.690100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ebcef-5db1-4ae5-8a45-72a199a34bb0 admin admin] PATH_INFO: `/v3/projects/9920a2a7a53b4be5ba5fd70946f6f083/users/ff45383a8f2f467f9232d7d5380f4908/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.692086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ebcef-5db1-4ae5-8a45-72a199a34bb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9920a2a7a53b4be5ba5fd70946f6f083, user_id=ff45383a8f2f467f9232d7d5380f4908, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.694726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ebcef-5db1-4ae5-8a45-72a199a34bb0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.704942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1236/2485] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:02 2022] PUT /identity/v3/projects/9920a2a7a53b4be5ba5fd70946f6f083/users/ff45383a8f2f467f9232d7d5380f4908/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:02.714782 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa4ff1-35d3-4663-be9c-0cae492c495d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.715228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa4ff1-35d3-4663-be9c-0cae492c495d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.715612 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa4ff1-35d3-4663-be9c-0cae492c495d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.742226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-92aa4ff1-35d3-4663-be9c-0cae492c495d None None] MFA Rules not processed for user `ff45383a8f2f467f9232d7d5380f4908`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:02.743498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-92aa4ff1-35d3-4663-be9c-0cae492c495d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:02.791040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:02.816298 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1250/2486] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:02.828639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ebcef-5db1-4ae5-8a45-72a199a34bb0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.830441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1c2ebcef-5db1-4ae5-8a45-72a199a34bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:02.891415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ebcef-5db1-4ae5-8a45-72a199a34bb0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.895096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46a9de-f7b1-4cde-9bf6-76b138bee453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.895783 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46a9de-f7b1-4cde-9bf6-76b138bee453 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.896077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46a9de-f7b1-4cde-9bf6-76b138bee453 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.896386 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46a9de-f7b1-4cde-9bf6-76b138bee453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.906051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46a9de-f7b1-4cde-9bf6-76b138bee453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.907457 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46a9de-f7b1-4cde-9bf6-76b138bee453 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.918400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1237/2487] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:02.998022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa4ff1-35d3-4663-be9c-0cae492c495d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.012711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa4ff1-35d3-4663-be9c-0cae492c495d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.020677 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589baa93-5ddc-4006-bff7-edfa8b348641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.021331 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589baa93-5ddc-4006-bff7-edfa8b348641 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.021614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589baa93-5ddc-4006-bff7-edfa8b348641 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.021891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589baa93-5ddc-4006-bff7-edfa8b348641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.080937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589baa93-5ddc-4006-bff7-edfa8b348641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.082375 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589baa93-5ddc-4006-bff7-edfa8b348641 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.091642 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1251/2488] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:03.576932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46a9de-f7b1-4cde-9bf6-76b138bee453 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.578352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bb46a9de-f7b1-4cde-9bf6-76b138bee453 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:03.643466 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46a9de-f7b1-4cde-9bf6-76b138bee453 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.648063 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.648994 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.649274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.649547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] PATH_INFO: `/v3/users/58193a8749434527a019fe2f8d92ee90` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.650771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] RBAC: Authorizing `identity:delete_user(user_id=58193a8749434527a019fe2f8d92ee90)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.652718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.698990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58193a8749434527a019fe2f8d92ee90'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:03.699611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58193a8749434527a019fe2f8d92ee90'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:03.707320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58193a8749434527a019fe2f8d92ee90'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:03.719694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58193a8749434527a019fe2f8d92ee90'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:03.726266 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58193a8749434527a019fe2f8d92ee90'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:03.727687 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:03.733342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1238/2489] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:03 2022] DELETE /identity/v3/users/58193a8749434527a019fe2f8d92ee90 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:03.737810 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f9bfe-7680-43ef-b55a-02106cbd18ec None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.738278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f9bfe-7680-43ef-b55a-02106cbd18ec None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.738401 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f9bfe-7680-43ef-b55a-02106cbd18ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.755518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fd6f9bfe-7680-43ef-b55a-02106cbd18ec None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:03.756737 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd6f9bfe-7680-43ef-b55a-02106cbd18ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:03.815391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1252/2490] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:03.825488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.826937 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:03.884063 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc60b2-0566-4d70-8f9c-d4450c21aafd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.888344 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fe22d-267f-4969-b230-6f0ba88f1f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.888989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fe22d-267f-4969-b230-6f0ba88f1f78 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.889274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fe22d-267f-4969-b230-6f0ba88f1f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.889551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fe22d-267f-4969-b230-6f0ba88f1f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.898943 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fe22d-267f-4969-b230-6f0ba88f1f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.900407 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fe22d-267f-4969-b230-6f0ba88f1f78 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.910575 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1239/2491] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:03.981057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f9bfe-7680-43ef-b55a-02106cbd18ec None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.982503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd6f9bfe-7680-43ef-b55a-02106cbd18ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:04.040743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f9bfe-7680-43ef-b55a-02106cbd18ec None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:04.044856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420bc2c-b299-43b1-8e2e-22b4f5bca6f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:04.045505 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420bc2c-b299-43b1-8e2e-22b4f5bca6f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:04.045794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420bc2c-b299-43b1-8e2e-22b4f5bca6f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:04.046070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420bc2c-b299-43b1-8e2e-22b4f5bca6f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:04.047866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9420bc2c-b299-43b1-8e2e-22b4f5bca6f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:04.092333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:04.118981 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420bc2c-b299-43b1-8e2e-22b4f5bca6f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:04.120730 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420bc2c-b299-43b1-8e2e-22b4f5bca6f4 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:04.130856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1253/2492] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:04.191617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fe22d-267f-4969-b230-6f0ba88f1f78 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:04.193726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a8fe22d-267f-4969-b230-6f0ba88f1f78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:04.209074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fe22d-267f-4969-b230-6f0ba88f1f78 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:04.214698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:04.215567 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:04.215879 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:04.216319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8 admin admin] PATH_INFO: `/v3/projects/022a79ab105348b2bdb0ea9a33a7db5d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:04.217570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=022a79ab105348b2bdb0ea9a33a7db5d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:04.219472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:04.267922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '022a79ab105348b2bdb0ea9a33a7db5d', 'request_id': 'req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:04.276988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '022a79ab105348b2bdb0ea9a33a7db5d', 'request_id': 'req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:04.282997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '022a79ab105348b2bdb0ea9a33a7db5d', 'request_id': 'req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:04.295778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8 admin admin] The token cache is being invalidate because project 022a79ab105348b2bdb0ea9a33a7db5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:04.296094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:04.297374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9adcab0c-5bca-4d22-bc8c-2f1d599930c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:04.299140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1240/2493] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:04 2022] DELETE /identity/v3/projects/022a79ab105348b2bdb0ea9a33a7db5d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:10.986175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420bc2c-b299-43b1-8e2e-22b4f5bca6f4 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:10.987600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9420bc2c-b299-43b1-8e2e-22b4f5bca6f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:11.001332 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420bc2c-b299-43b1-8e2e-22b4f5bca6f4 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:11.056673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:11.058105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:11.058384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:11.058657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] PATH_INFO: `/v3/users/90e32bab72924abfa9f4bf2053ead72f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:11.059863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e32bab72924abfa9f4bf2053ead72f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:11.061791 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:11.108828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e32bab72924abfa9f4bf2053ead72f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:11.109618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e32bab72924abfa9f4bf2053ead72f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:11.110231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e32bab72924abfa9f4bf2053ead72f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:11.123033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e32bab72924abfa9f4bf2053ead72f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:11.130035 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e32bab72924abfa9f4bf2053ead72f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:11.143405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1254/2494] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:10 2022] DELETE /identity/v3/users/90e32bab72924abfa9f4bf2053ead72f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:11.147999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa54959-2ce8-4624-ba8c-76a89d54b31e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:11.148471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa54959-2ce8-4624-ba8c-76a89d54b31e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:11.148569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa54959-2ce8-4624-ba8c-76a89d54b31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:11.166896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-efa54959-2ce8-4624-ba8c-76a89d54b31e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:11.168146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-efa54959-2ce8-4624-ba8c-76a89d54b31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:11.228234 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1241/2495] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:11.238049 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:11.239454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:11.298748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc6178-6017-4ccb-a531-0ca6df4d70a7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:11.302598 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc49d0d-375e-451f-b04b-b10a92ad71f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:11.303243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc49d0d-375e-451f-b04b-b10a92ad71f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:11.303530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc49d0d-375e-451f-b04b-b10a92ad71f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:11.303888 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc49d0d-375e-451f-b04b-b10a92ad71f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:11.313752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc49d0d-375e-451f-b04b-b10a92ad71f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:11.315220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc49d0d-375e-451f-b04b-b10a92ad71f4 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:11.319315 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:11.329788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1255/2496] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:11.801080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa54959-2ce8-4624-ba8c-76a89d54b31e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:11.802563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-efa54959-2ce8-4624-ba8c-76a89d54b31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:11.815998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa54959-2ce8-4624-ba8c-76a89d54b31e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:11.820880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:11.822286 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:11.823096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:11.823551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] PATH_INFO: `/v3/projects/aabc22f4e29346bab822810573f531a7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:11.824801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aabc22f4e29346bab822810573f531a7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:11.826701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:11.848852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:11.876071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aabc22f4e29346bab822810573f531a7', 'request_id': 'req-dd907dfb-40ab-4cf8-87bd-276232fc07d1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:11.884300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aabc22f4e29346bab822810573f531a7', 'request_id': 'req-dd907dfb-40ab-4cf8-87bd-276232fc07d1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:11.890582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aabc22f4e29346bab822810573f531a7', 'request_id': 'req-dd907dfb-40ab-4cf8-87bd-276232fc07d1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:11.902445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] The token cache is being invalidate because project aabc22f4e29346bab822810573f531a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:11.902743 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:11.903403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:11.905229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1242/2497] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:11 2022] DELETE /identity/v3/projects/aabc22f4e29346bab822810573f531a7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:11.908866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc49d0d-375e-451f-b04b-b10a92ad71f4 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:11.922813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc49d0d-375e-451f-b04b-b10a92ad71f4 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:11.988113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e327e3-e7a4-499a-bce0-d2e1c85623be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:11.989006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e327e3-e7a4-499a-bce0-d2e1c85623be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:11.989291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e327e3-e7a4-499a-bce0-d2e1c85623be admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:11.989569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e327e3-e7a4-499a-bce0-d2e1c85623be admin admin] PATH_INFO: `/v3/users/ff45383a8f2f467f9232d7d5380f4908` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:11.990794 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e327e3-e7a4-499a-bce0-d2e1c85623be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff45383a8f2f467f9232d7d5380f4908)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:11.992720 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e327e3-e7a4-499a-bce0-d2e1c85623be admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:12.040362 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06e327e3-e7a4-499a-bce0-d2e1c85623be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff45383a8f2f467f9232d7d5380f4908'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:12.040974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06e327e3-e7a4-499a-bce0-d2e1c85623be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff45383a8f2f467f9232d7d5380f4908'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:12.042139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06e327e3-e7a4-499a-bce0-d2e1c85623be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff45383a8f2f467f9232d7d5380f4908'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:12.053943 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06e327e3-e7a4-499a-bce0-d2e1c85623be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff45383a8f2f467f9232d7d5380f4908'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:12.060742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06e327e3-e7a4-499a-bce0-d2e1c85623be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff45383a8f2f467f9232d7d5380f4908'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:12.076203 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1256/2498] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:11 2022] DELETE /identity/v3/users/ff45383a8f2f467f9232d7d5380f4908 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:12.342841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:12.344383 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:12.357955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd907dfb-40ab-4cf8-87bd-276232fc07d1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:12.414694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a541c-16be-49fd-85f1-44b25162ec1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:12.415590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a541c-16be-49fd-85f1-44b25162ec1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:12.415903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a541c-16be-49fd-85f1-44b25162ec1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:12.416182 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a541c-16be-49fd-85f1-44b25162ec1b admin admin] PATH_INFO: `/v3/projects/9920a2a7a53b4be5ba5fd70946f6f083` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:12.417986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a541c-16be-49fd-85f1-44b25162ec1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9920a2a7a53b4be5ba5fd70946f6f083)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:12.419916 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a541c-16be-49fd-85f1-44b25162ec1b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:12.464349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-711a541c-16be-49fd-85f1-44b25162ec1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9920a2a7a53b4be5ba5fd70946f6f083', 'request_id': 'req-711a541c-16be-49fd-85f1-44b25162ec1b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:12.471419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-711a541c-16be-49fd-85f1-44b25162ec1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9920a2a7a53b4be5ba5fd70946f6f083', 'request_id': 'req-711a541c-16be-49fd-85f1-44b25162ec1b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:12.476886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-711a541c-16be-49fd-85f1-44b25162ec1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9920a2a7a53b4be5ba5fd70946f6f083', 'request_id': 'req-711a541c-16be-49fd-85f1-44b25162ec1b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:12.488223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-711a541c-16be-49fd-85f1-44b25162ec1b admin admin] The token cache is being invalidate because project 9920a2a7a53b4be5ba5fd70946f6f083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:12.488512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-711a541c-16be-49fd-85f1-44b25162ec1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:12.489141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-711a541c-16be-49fd-85f1-44b25162ec1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:12.490771 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1243/2499] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:12 2022] DELETE /identity/v3/projects/9920a2a7a53b4be5ba5fd70946f6f083 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:12.495704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9dadf-90ce-4de4-8db4-455872858951 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:12.496493 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9dadf-90ce-4de4-8db4-455872858951 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:12.496774 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9dadf-90ce-4de4-8db4-455872858951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:12.514961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f9e9dadf-90ce-4de4-8db4-455872858951 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:12.516447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f9e9dadf-90ce-4de4-8db4-455872858951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:12.573580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1257/2500] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:12.581038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e0f9c-3225-4cdb-b6e8-162a8951f325 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:12.581335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e0f9c-3225-4cdb-b6e8-162a8951f325 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:12.581619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e0f9c-3225-4cdb-b6e8-162a8951f325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:12.603535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1f3e0f9c-3225-4cdb-b6e8-162a8951f325 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:12.605272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1f3e0f9c-3225-4cdb-b6e8-162a8951f325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:12.613079 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1244/2501] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:12.617214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9dadf-90ce-4de4-8db4-455872858951 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:12.627516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9dadf-90ce-4de4-8db4-455872858951 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:12.632048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac44111-06e8-4887-aa9a-d09e1fca52f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:12.632694 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac44111-06e8-4887-aa9a-d09e1fca52f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:12.632973 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac44111-06e8-4887-aa9a-d09e1fca52f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:12.633246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac44111-06e8-4887-aa9a-d09e1fca52f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:12.633962 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac44111-06e8-4887-aa9a-d09e1fca52f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:12.634269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac44111-06e8-4887-aa9a-d09e1fca52f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:12.636431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac44111-06e8-4887-aa9a-d09e1fca52f7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:12.647998 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1258/2502] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:12.652579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e0f9c-3225-4cdb-b6e8-162a8951f325 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:12.661626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e0f9c-3225-4cdb-b6e8-162a8951f325 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:12.666233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff7c8e-fadb-4573-9c6b-42107155e469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:12.666850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff7c8e-fadb-4573-9c6b-42107155e469 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:12.667127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff7c8e-fadb-4573-9c6b-42107155e469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:12.667396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff7c8e-fadb-4573-9c6b-42107155e469 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:12.668117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff7c8e-fadb-4573-9c6b-42107155e469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:12.669985 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff7c8e-fadb-4573-9c6b-42107155e469 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:12.694656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1245/2503] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:12 2022] POST /identity/v3/projects => generated 375 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:12.699083 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac44111-06e8-4887-aa9a-d09e1fca52f7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:12.705505 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:12.709319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac44111-06e8-4887-aa9a-d09e1fca52f7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:12.767255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ab2d5-4549-4d28-bd35-5b78d8d2e6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:12.767954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ab2d5-4549-4d28-bd35-5b78d8d2e6af admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:12.768236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ab2d5-4549-4d28-bd35-5b78d8d2e6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:12.768509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ab2d5-4549-4d28-bd35-5b78d8d2e6af admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:12.769192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ab2d5-4549-4d28-bd35-5b78d8d2e6af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:12.771092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ab2d5-4549-4d28-bd35-5b78d8d2e6af admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:12.790219 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1259/2504] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:12 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:12.794333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff7c8e-fadb-4573-9c6b-42107155e469 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:12.805870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff7c8e-fadb-4573-9c6b-42107155e469 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:12.811511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8faedc7-8d21-46a1-89fd-df8ea705b7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:12.812167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faedc7-8d21-46a1-89fd-df8ea705b7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:12.812450 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faedc7-8d21-46a1-89fd-df8ea705b7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:12.812725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faedc7-8d21-46a1-89fd-df8ea705b7b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:12.813654 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8faedc7-8d21-46a1-89fd-df8ea705b7b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:12.815544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8faedc7-8d21-46a1-89fd-df8ea705b7b1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:12.818350 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:12.830945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1246/2505] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:12.834677 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ab2d5-4549-4d28-bd35-5b78d8d2e6af admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:12.845519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ab2d5-4549-4d28-bd35-5b78d8d2e6af admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:12.850829 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb0a34-7000-4186-a780-150fd1a7c24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:12.851489 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb0a34-7000-4186-a780-150fd1a7c24b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:12.851839 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb0a34-7000-4186-a780-150fd1a7c24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:12.852117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb0a34-7000-4186-a780-150fd1a7c24b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:12.852846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb0a34-7000-4186-a780-150fd1a7c24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:12.855019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb0a34-7000-4186-a780-150fd1a7c24b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:12.866249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1260/2506] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:12.870211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8faedc7-8d21-46a1-89fd-df8ea705b7b1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:12.880890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8faedc7-8d21-46a1-89fd-df8ea705b7b1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:12.886358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694ee80-a9c5-4124-a8cb-f79f30246f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:12.887272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694ee80-a9c5-4124-a8cb-f79f30246f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:12.887555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694ee80-a9c5-4124-a8cb-f79f30246f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:12.887864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694ee80-a9c5-4124-a8cb-f79f30246f33 admin admin] PATH_INFO: `/v3/projects/376c648240b44246a838156c31df3fbf/users/ed7a7571f9c648f8a74bf3829bf2c5b6/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:12.905332 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694ee80-a9c5-4124-a8cb-f79f30246f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376c648240b44246a838156c31df3fbf, user_id=ed7a7571f9c648f8a74bf3829bf2c5b6, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:12.908290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694ee80-a9c5-4124-a8cb-f79f30246f33 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:12.923204 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1247/2507] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:12 2022] PUT /identity/v3/projects/376c648240b44246a838156c31df3fbf/users/ed7a7571f9c648f8a74bf3829bf2c5b6/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:12.935206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624696c-8209-4ac6-aaba-cc5c13a7d526 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:12.935500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624696c-8209-4ac6-aaba-cc5c13a7d526 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:12.935914 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624696c-8209-4ac6-aaba-cc5c13a7d526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:12.965236 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d624696c-8209-4ac6-aaba-cc5c13a7d526 None None] MFA Rules not processed for user `ed7a7571f9c648f8a74bf3829bf2c5b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:12.965669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d624696c-8209-4ac6-aaba-cc5c13a7d526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:13.039789 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1261/2508] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:13.051165 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694ee80-a9c5-4124-a8cb-f79f30246f33 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:13.052993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6694ee80-a9c5-4124-a8cb-f79f30246f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:13.115887 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694ee80-a9c5-4124-a8cb-f79f30246f33 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:13.120472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b2772-06bf-4506-a707-8247a8dfc2b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af6cc8c56db48beae2550954446c119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:13.121166 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b2772-06bf-4506-a707-8247a8dfc2b2 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:13.121448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b2772-06bf-4506-a707-8247a8dfc2b2 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:13.121725 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b2772-06bf-4506-a707-8247a8dfc2b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:13.130401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b2772-06bf-4506-a707-8247a8dfc2b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:13.131877 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b2772-06bf-4506-a707-8247a8dfc2b2 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:13.142462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1248/2509] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:14.130599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1f10-52d9-44ad-9fec-8c740a501118 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:14.131139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1f10-52d9-44ad-9fec-8c740a501118 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:14.131240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1f10-52d9-44ad-9fec-8c740a501118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:14.150530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2bed1f10-52d9-44ad-9fec-8c740a501118 None None] MFA Rules not processed for user `8f8da2142b9e4751b436cfd4920587a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:14.152377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2bed1f10-52d9-44ad-9fec-8c740a501118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:14.207734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1262/2510] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:23.140823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:24.200970 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:35.374793 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b2772-06bf-4506-a707-8247a8dfc2b2 service glance] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:35.376345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-485b2772-06bf-4506-a707-8247a8dfc2b2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:35.431577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b2772-06bf-4506-a707-8247a8dfc2b2 service glance] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:35.435142 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400060d-4732-4939-a6f9-371c991cd058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:35.436332 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400060d-4732-4939-a6f9-371c991cd058 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:35.436619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400060d-4732-4939-a6f9-371c991cd058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:35.436898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400060d-4732-4939-a6f9-371c991cd058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:35.438621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d400060d-4732-4939-a6f9-371c991cd058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:35.499891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400060d-4732-4939-a6f9-371c991cd058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:35.501310 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400060d-4732-4939-a6f9-371c991cd058 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:35.510798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1249/2511] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:37.055306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed1f10-52d9-44ad-9fec-8c740a501118 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:37.119487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed1f10-52d9-44ad-9fec-8c740a501118 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:37.129088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:37.129979 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:37.130264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:37.130541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] PATH_INFO: `/v3/users/ed7a7571f9c648f8a74bf3829bf2c5b6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:37.131776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7a7571f9c648f8a74bf3829bf2c5b6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:37.133913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:37.183005 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7a7571f9c648f8a74bf3829bf2c5b6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.183686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7a7571f9c648f8a74bf3829bf2c5b6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.184305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7a7571f9c648f8a74bf3829bf2c5b6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.196563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7a7571f9c648f8a74bf3829bf2c5b6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.205545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7a7571f9c648f8a74bf3829bf2c5b6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.223039 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1263/2512] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:37 2022] DELETE /identity/v3/users/ed7a7571f9c648f8a74bf3829bf2c5b6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:37.227683 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21febbab-353a-4d42-b30d-5720768d4dd4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:37.228157 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21febbab-353a-4d42-b30d-5720768d4dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:37.228255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21febbab-353a-4d42-b30d-5720768d4dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:37.246585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-21febbab-353a-4d42-b30d-5720768d4dd4 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:37.247779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-21febbab-353a-4d42-b30d-5720768d4dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:37.323445 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1250/2513] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:37.330882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:37.332335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:37.418999 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802a2dd-7cb5-431d-b9c6-bf856a75ff64 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:37.425747 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e176a5b-2f6d-4aaa-a71f-34ae29b0c5a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:37.426806 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176a5b-2f6d-4aaa-a71f-34ae29b0c5a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:37.427269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176a5b-2f6d-4aaa-a71f-34ae29b0c5a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:37.427944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176a5b-2f6d-4aaa-a71f-34ae29b0c5a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:37.445237 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e176a5b-2f6d-4aaa-a71f-34ae29b0c5a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:37.446765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e176a5b-2f6d-4aaa-a71f-34ae29b0c5a4 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:37.471745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1264/2514] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:37.884319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21febbab-353a-4d42-b30d-5720768d4dd4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:37.885930 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-21febbab-353a-4d42-b30d-5720768d4dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:37.900724 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21febbab-353a-4d42-b30d-5720768d4dd4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:37.905972 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541b283-80e0-40c2-ab15-6d461a7a4e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:37.907412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541b283-80e0-40c2-ab15-6d461a7a4e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:37.907412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541b283-80e0-40c2-ab15-6d461a7a4e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:37.907681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541b283-80e0-40c2-ab15-6d461a7a4e4b admin admin] PATH_INFO: `/v3/projects/376c648240b44246a838156c31df3fbf` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:37.908980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541b283-80e0-40c2-ab15-6d461a7a4e4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=376c648240b44246a838156c31df3fbf)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:37.911379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541b283-80e0-40c2-ab15-6d461a7a4e4b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:37.973222 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:37.977317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a541b283-80e0-40c2-ab15-6d461a7a4e4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '376c648240b44246a838156c31df3fbf', 'request_id': 'req-a541b283-80e0-40c2-ab15-6d461a7a4e4b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.986819 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a541b283-80e0-40c2-ab15-6d461a7a4e4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '376c648240b44246a838156c31df3fbf', 'request_id': 'req-a541b283-80e0-40c2-ab15-6d461a7a4e4b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.992952 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a541b283-80e0-40c2-ab15-6d461a7a4e4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '376c648240b44246a838156c31df3fbf', 'request_id': 'req-a541b283-80e0-40c2-ab15-6d461a7a4e4b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:38.004985 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a541b283-80e0-40c2-ab15-6d461a7a4e4b admin admin] The token cache is being invalidate because project 376c648240b44246a838156c31df3fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:38.005411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a541b283-80e0-40c2-ab15-6d461a7a4e4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:38.006153 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a541b283-80e0-40c2-ab15-6d461a7a4e4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:38.007794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1251/2515] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:37 2022] DELETE /identity/v3/projects/376c648240b44246a838156c31df3fbf => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:38.012634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8da3a9-7187-489a-bb70-eecaeb92a630 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.012924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8da3a9-7187-489a-bb70-eecaeb92a630 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.013204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8da3a9-7187-489a-bb70-eecaeb92a630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.031975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ce8da3a9-7187-489a-bb70-eecaeb92a630 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:38.033115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ce8da3a9-7187-489a-bb70-eecaeb92a630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:38.050568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:38.099105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1265/2516] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:38.107507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ca21e-f94b-46fb-b2c9-dac9a936e457 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.107927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ca21e-f94b-46fb-b2c9-dac9a936e457 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.108209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ca21e-f94b-46fb-b2c9-dac9a936e457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.130272 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fa8ca21e-f94b-46fb-b2c9-dac9a936e457 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:38.131527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fa8ca21e-f94b-46fb-b2c9-dac9a936e457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:38.140514 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1252/2517] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:38.144997 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8da3a9-7187-489a-bb70-eecaeb92a630 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.155360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8da3a9-7187-489a-bb70-eecaeb92a630 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.160228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa34bdc-9d5a-4830-97c3-f2a1c741f0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.160903 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa34bdc-9d5a-4830-97c3-f2a1c741f0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.161191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa34bdc-9d5a-4830-97c3-f2a1c741f0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.161473 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa34bdc-9d5a-4830-97c3-f2a1c741f0ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.162244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa34bdc-9d5a-4830-97c3-f2a1c741f0ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:38.162558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa34bdc-9d5a-4830-97c3-f2a1c741f0ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.164202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa34bdc-9d5a-4830-97c3-f2a1c741f0ce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.176709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1266/2518] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:10:38.181314 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ca21e-f94b-46fb-b2c9-dac9a936e457 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.191609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ca21e-f94b-46fb-b2c9-dac9a936e457 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.196498 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0282ae8-6c18-4196-9cf2-ecae6e8e8354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.197235 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0282ae8-6c18-4196-9cf2-ecae6e8e8354 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.197750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0282ae8-6c18-4196-9cf2-ecae6e8e8354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.197997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0282ae8-6c18-4196-9cf2-ecae6e8e8354 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.198775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0282ae8-6c18-4196-9cf2-ecae6e8e8354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.200993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0282ae8-6c18-4196-9cf2-ecae6e8e8354 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.230467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1253/2519] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:10:38.231699 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa34bdc-9d5a-4830-97c3-f2a1c741f0ce admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.242644 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa34bdc-9d5a-4830-97c3-f2a1c741f0ce admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.303349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c441d8-bf85-4f10-aad6-07119c1dd40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.303994 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c441d8-bf85-4f10-aad6-07119c1dd40b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.304276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c441d8-bf85-4f10-aad6-07119c1dd40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.304633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c441d8-bf85-4f10-aad6-07119c1dd40b admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.305325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c441d8-bf85-4f10-aad6-07119c1dd40b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.307240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c441d8-bf85-4f10-aad6-07119c1dd40b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.328493 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1267/2520] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:10:38.333831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0282ae8-6c18-4196-9cf2-ecae6e8e8354 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.344475 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0282ae8-6c18-4196-9cf2-ecae6e8e8354 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.349485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab07b55-1872-4e25-8500-51f23c04f00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.350143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab07b55-1872-4e25-8500-51f23c04f00c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.350428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab07b55-1872-4e25-8500-51f23c04f00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.350704 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab07b55-1872-4e25-8500-51f23c04f00c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.351432 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab07b55-1872-4e25-8500-51f23c04f00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.353057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab07b55-1872-4e25-8500-51f23c04f00c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.364430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1254/2521] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:38.368795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c441d8-bf85-4f10-aad6-07119c1dd40b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.379669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c441d8-bf85-4f10-aad6-07119c1dd40b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.384474 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435356d-ce57-4b7e-bdf7-079d34ba78b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.385211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435356d-ce57-4b7e-bdf7-079d34ba78b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.385582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435356d-ce57-4b7e-bdf7-079d34ba78b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.385902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435356d-ce57-4b7e-bdf7-079d34ba78b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.386695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435356d-ce57-4b7e-bdf7-079d34ba78b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.388500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435356d-ce57-4b7e-bdf7-079d34ba78b3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.399010 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1268/2522] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:38.403273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab07b55-1872-4e25-8500-51f23c04f00c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.415192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab07b55-1872-4e25-8500-51f23c04f00c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.420528 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3dbee-3300-45c8-94e1-ca413fcedb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.421496 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3dbee-3300-45c8-94e1-ca413fcedb72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.421775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3dbee-3300-45c8-94e1-ca413fcedb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.422201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3dbee-3300-45c8-94e1-ca413fcedb72 admin admin] PATH_INFO: `/v3/projects/5b5e7aba22dd4bf6992cb37605e50a11/users/923bcaa783814db2b09440356d8a90ff/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.438192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3dbee-3300-45c8-94e1-ca413fcedb72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b5e7aba22dd4bf6992cb37605e50a11, user_id=923bcaa783814db2b09440356d8a90ff, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.440938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3dbee-3300-45c8-94e1-ca413fcedb72 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.455091 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1255/2523] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:38 2022] PUT /identity/v3/projects/5b5e7aba22dd4bf6992cb37605e50a11/users/923bcaa783814db2b09440356d8a90ff/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:38.464975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e0078-6c50-475b-883b-8e57bbd5a647 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.465295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e0078-6c50-475b-883b-8e57bbd5a647 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.465575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e0078-6c50-475b-883b-8e57bbd5a647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.491932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-608e0078-6c50-475b-883b-8e57bbd5a647 None None] MFA Rules not processed for user `923bcaa783814db2b09440356d8a90ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:38.493136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-608e0078-6c50-475b-883b-8e57bbd5a647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:38.556458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1269/2524] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:38.561084 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3dbee-3300-45c8-94e1-ca413fcedb72 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.621160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3dbee-3300-45c8-94e1-ca413fcedb72 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.626126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d27dd-f41e-4f1f-ad86-4661ce1d2702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.626778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d27dd-f41e-4f1f-ad86-4661ce1d2702 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.627065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d27dd-f41e-4f1f-ad86-4661ce1d2702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.627337 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d27dd-f41e-4f1f-ad86-4661ce1d2702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.628075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d27dd-f41e-4f1f-ad86-4661ce1d2702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.629940 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d27dd-f41e-4f1f-ad86-4661ce1d2702 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.656324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1256/2525] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:38.660492 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e0078-6c50-475b-883b-8e57bbd5a647 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.672432 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e0078-6c50-475b-883b-8e57bbd5a647 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.685554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:38.729977 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47721f-300e-42d3-805d-f2911b1ccde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.730655 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47721f-300e-42d3-805d-f2911b1ccde6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.730944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47721f-300e-42d3-805d-f2911b1ccde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.731227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47721f-300e-42d3-805d-f2911b1ccde6 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.731967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47721f-300e-42d3-805d-f2911b1ccde6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.734515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47721f-300e-42d3-805d-f2911b1ccde6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.753984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1270/2526] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:38.757958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d27dd-f41e-4f1f-ad86-4661ce1d2702 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.768618 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d27dd-f41e-4f1f-ad86-4661ce1d2702 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.773193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687b523-96db-45f8-bf3c-21aa677b2612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.773845 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687b523-96db-45f8-bf3c-21aa677b2612 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.774128 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687b523-96db-45f8-bf3c-21aa677b2612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.774406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687b523-96db-45f8-bf3c-21aa677b2612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.775127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687b523-96db-45f8-bf3c-21aa677b2612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.776745 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687b523-96db-45f8-bf3c-21aa677b2612 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.787633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1257/2527] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:38.791755 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47721f-300e-42d3-805d-f2911b1ccde6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.802320 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47721f-300e-42d3-805d-f2911b1ccde6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.807000 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e79e92-84bc-4fef-bd0e-15cd7075870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.807592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e79e92-84bc-4fef-bd0e-15cd7075870c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.807913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e79e92-84bc-4fef-bd0e-15cd7075870c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.808197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e79e92-84bc-4fef-bd0e-15cd7075870c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.808930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e79e92-84bc-4fef-bd0e-15cd7075870c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.810528 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e79e92-84bc-4fef-bd0e-15cd7075870c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.820225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1271/2528] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:38.824188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687b523-96db-45f8-bf3c-21aa677b2612 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.834098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687b523-96db-45f8-bf3c-21aa677b2612 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.838773 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2664b-90fd-4b34-af1a-0da882c15cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.839643 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2664b-90fd-4b34-af1a-0da882c15cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.839963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2664b-90fd-4b34-af1a-0da882c15cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.840243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2664b-90fd-4b34-af1a-0da882c15cc7 admin admin] PATH_INFO: `/v3/projects/83d0bf54d6fd4af1a1ee8038e1a8cf17/users/e260e63c9ac2484b80b74a9052d00e44/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.855077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2664b-90fd-4b34-af1a-0da882c15cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83d0bf54d6fd4af1a1ee8038e1a8cf17, user_id=e260e63c9ac2484b80b74a9052d00e44, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.858382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2664b-90fd-4b34-af1a-0da882c15cc7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.870860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1258/2529] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:38 2022] PUT /identity/v3/projects/83d0bf54d6fd4af1a1ee8038e1a8cf17/users/e260e63c9ac2484b80b74a9052d00e44/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:38.880906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb251b-9b9b-4059-b756-007c31de36c6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.881206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb251b-9b9b-4059-b756-007c31de36c6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.881490 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb251b-9b9b-4059-b756-007c31de36c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.908730 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f8bb251b-9b9b-4059-b756-007c31de36c6 None None] MFA Rules not processed for user `e260e63c9ac2484b80b74a9052d00e44`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:38.909980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f8bb251b-9b9b-4059-b756-007c31de36c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:38.976541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1272/2530] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:38.981298 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2664b-90fd-4b34-af1a-0da882c15cc7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.002507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:39.048136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2664b-90fd-4b34-af1a-0da882c15cc7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.052813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6e648-cf9f-4b78-817b-bd7ba113681a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.053465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6e648-cf9f-4b78-817b-bd7ba113681a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.053746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6e648-cf9f-4b78-817b-bd7ba113681a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.054020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6e648-cf9f-4b78-817b-bd7ba113681a admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.054699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6e648-cf9f-4b78-817b-bd7ba113681a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.056634 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6e648-cf9f-4b78-817b-bd7ba113681a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.081695 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1259/2531] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:39.085763 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb251b-9b9b-4059-b756-007c31de36c6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.097347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb251b-9b9b-4059-b756-007c31de36c6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.152669 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d95bc-66a1-41f9-80e3-59078d6699cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.153342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d95bc-66a1-41f9-80e3-59078d6699cb admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.153618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d95bc-66a1-41f9-80e3-59078d6699cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.153893 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d95bc-66a1-41f9-80e3-59078d6699cb admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.155135 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d95bc-66a1-41f9-80e3-59078d6699cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.157105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d95bc-66a1-41f9-80e3-59078d6699cb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.177018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1273/2532] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:39 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:39.180993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6e648-cf9f-4b78-817b-bd7ba113681a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.191210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6e648-cf9f-4b78-817b-bd7ba113681a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.196082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957ca37-8ca0-42b0-b439-cc583fe5fc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.196732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957ca37-8ca0-42b0-b439-cc583fe5fc10 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.197014 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957ca37-8ca0-42b0-b439-cc583fe5fc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.197287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957ca37-8ca0-42b0-b439-cc583fe5fc10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.198008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957ca37-8ca0-42b0-b439-cc583fe5fc10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.199591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957ca37-8ca0-42b0-b439-cc583fe5fc10 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.209868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1260/2533] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:39.213788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d95bc-66a1-41f9-80e3-59078d6699cb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.224206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d95bc-66a1-41f9-80e3-59078d6699cb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.228844 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4b170-4244-4c08-8299-b08882df7a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.229712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4b170-4244-4c08-8299-b08882df7a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.229996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4b170-4244-4c08-8299-b08882df7a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.230275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4b170-4244-4c08-8299-b08882df7a52 admin admin] PATH_INFO: `/v3/projects/e316efb172de48a685718fc384b5d5e3/users/79727dc36ac9424fb2a8b8afaba006e1/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.243153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:39.246532 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4b170-4244-4c08-8299-b08882df7a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e316efb172de48a685718fc384b5d5e3, user_id=79727dc36ac9424fb2a8b8afaba006e1, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.250324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4b170-4244-4c08-8299-b08882df7a52 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.265420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1274/2534] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:39 2022] PUT /identity/v3/projects/e316efb172de48a685718fc384b5d5e3/users/79727dc36ac9424fb2a8b8afaba006e1/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:39.269533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957ca37-8ca0-42b0-b439-cc583fe5fc10 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.279934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957ca37-8ca0-42b0-b439-cc583fe5fc10 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.337485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6befcd-5606-4ff0-809d-539a9b370315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.338074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6befcd-5606-4ff0-809d-539a9b370315 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.338355 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6befcd-5606-4ff0-809d-539a9b370315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.338627 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6befcd-5606-4ff0-809d-539a9b370315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.339347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6befcd-5606-4ff0-809d-539a9b370315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.340963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6befcd-5606-4ff0-809d-539a9b370315 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.350407 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1261/2535] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:39.354550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4b170-4244-4c08-8299-b08882df7a52 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.364630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4b170-4244-4c08-8299-b08882df7a52 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.369121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5f384-28cf-4d6e-9dc5-5ca73003e714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.369986 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5f384-28cf-4d6e-9dc5-5ca73003e714 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.370265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5f384-28cf-4d6e-9dc5-5ca73003e714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.370541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5f384-28cf-4d6e-9dc5-5ca73003e714 admin admin] PATH_INFO: `/v3/projects/e316efb172de48a685718fc384b5d5e3/users/79727dc36ac9424fb2a8b8afaba006e1/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.372535 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5f384-28cf-4d6e-9dc5-5ca73003e714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e316efb172de48a685718fc384b5d5e3, user_id=79727dc36ac9424fb2a8b8afaba006e1, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.375649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5f384-28cf-4d6e-9dc5-5ca73003e714 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.385515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1275/2536] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:39 2022] PUT /identity/v3/projects/e316efb172de48a685718fc384b5d5e3/users/79727dc36ac9424fb2a8b8afaba006e1/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:39.397013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d8558-83e3-47a1-9922-722f37ec6688 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.397421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d8558-83e3-47a1-9922-722f37ec6688 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.397688 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d8558-83e3-47a1-9922-722f37ec6688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.423087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c28d8558-83e3-47a1-9922-722f37ec6688 None None] MFA Rules not processed for user `79727dc36ac9424fb2a8b8afaba006e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:39.424320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c28d8558-83e3-47a1-9922-722f37ec6688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:39.490250 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1262/2537] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:39.501988 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5f384-28cf-4d6e-9dc5-5ca73003e714 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.503753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-61f5f384-28cf-4d6e-9dc5-5ca73003e714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:39.559168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5f384-28cf-4d6e-9dc5-5ca73003e714 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.563337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8690a0d-4c3e-49a2-a5ad-0ba303bcc556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.564006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8690a0d-4c3e-49a2-a5ad-0ba303bcc556 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.564298 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8690a0d-4c3e-49a2-a5ad-0ba303bcc556 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.564579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8690a0d-4c3e-49a2-a5ad-0ba303bcc556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.618807 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8690a0d-4c3e-49a2-a5ad-0ba303bcc556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.620228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8690a0d-4c3e-49a2-a5ad-0ba303bcc556 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.630487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1276/2538] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:42.155696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d8558-83e3-47a1-9922-722f37ec6688 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:42.157380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c28d8558-83e3-47a1-9922-722f37ec6688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:42.170625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d8558-83e3-47a1-9922-722f37ec6688 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:42.174955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c385a4-0b8a-4784-a731-0443f8911e74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:42.175597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c385a4-0b8a-4784-a731-0443f8911e74 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:42.175911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c385a4-0b8a-4784-a731-0443f8911e74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:42.176196 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c385a4-0b8a-4784-a731-0443f8911e74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:42.188773 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:42.232862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c385a4-0b8a-4784-a731-0443f8911e74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:42.234796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c385a4-0b8a-4784-a731-0443f8911e74 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:42.243783 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1263/2539] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:42.757056 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8690a0d-4c3e-49a2-a5ad-0ba303bcc556 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:42.765594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8690a0d-4c3e-49a2-a5ad-0ba303bcc556 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:42.769394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d0b7b-1f9f-4b64-9191-6b6f8a889050 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:42.770080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d0b7b-1f9f-4b64-9191-6b6f8a889050 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:42.770306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d0b7b-1f9f-4b64-9191-6b6f8a889050 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:42.770579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d0b7b-1f9f-4b64-9191-6b6f8a889050 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:42.779347 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d0b7b-1f9f-4b64-9191-6b6f8a889050 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:42.780870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d0b7b-1f9f-4b64-9191-6b6f8a889050 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:42.790087 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1277/2540] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:44.694101 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c385a4-0b8a-4784-a731-0443f8911e74 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:44.695586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-62c385a4-0b8a-4784-a731-0443f8911e74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:44.760752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c385a4-0b8a-4784-a731-0443f8911e74 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:44.766211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:44.767108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:44.767381 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:44.767676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] PATH_INFO: `/v3/users/063702961afa471e81861e622a06b0f9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:44.769276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=063702961afa471e81861e622a06b0f9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:44.771245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:44.816837 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '063702961afa471e81861e622a06b0f9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:44.817503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '063702961afa471e81861e622a06b0f9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:44.824717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '063702961afa471e81861e622a06b0f9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:44.837598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '063702961afa471e81861e622a06b0f9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:44.844331 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '063702961afa471e81861e622a06b0f9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:44.850155 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1264/2541] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:44 2022] DELETE /identity/v3/users/063702961afa471e81861e622a06b0f9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:44.854872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2632eb3-bbee-48ed-af57-3baecd28ad4b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:44.855341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2632eb3-bbee-48ed-af57-3baecd28ad4b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:44.855454 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2632eb3-bbee-48ed-af57-3baecd28ad4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:44.872006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e2632eb3-bbee-48ed-af57-3baecd28ad4b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:44.873203 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e2632eb3-bbee-48ed-af57-3baecd28ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:44.931403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1278/2542] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:44.941419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:44.942830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:44.977131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:45.002668 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba55c79-350c-4ab2-a4a8-26fe97d176cd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:45.006351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40fa6f6-ac12-49ca-bb0b-497cc7d3668a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:45.006988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fa6f6-ac12-49ca-bb0b-497cc7d3668a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:45.007270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fa6f6-ac12-49ca-bb0b-497cc7d3668a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:45.007546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fa6f6-ac12-49ca-bb0b-497cc7d3668a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:45.016613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40fa6f6-ac12-49ca-bb0b-497cc7d3668a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:45.018219 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40fa6f6-ac12-49ca-bb0b-497cc7d3668a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:45.027551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1265/2543] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:45.301105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2632eb3-bbee-48ed-af57-3baecd28ad4b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:45.303080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e2632eb3-bbee-48ed-af57-3baecd28ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:45.322919 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2632eb3-bbee-48ed-af57-3baecd28ad4b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:45.328902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:45.329812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:45.330051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:45.330349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] PATH_INFO: `/v3/projects/12936b8fa28f4bebb77f3689874d24ac` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:45.331523 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12936b8fa28f4bebb77f3689874d24ac)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:45.333648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:45.351504 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:45.386572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12936b8fa28f4bebb77f3689874d24ac', 'request_id': 'req-5eae9343-e81a-4777-8187-e60cb7b836a2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:45.392707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12936b8fa28f4bebb77f3689874d24ac', 'request_id': 'req-5eae9343-e81a-4777-8187-e60cb7b836a2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:45.398587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12936b8fa28f4bebb77f3689874d24ac', 'request_id': 'req-5eae9343-e81a-4777-8187-e60cb7b836a2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:45.411539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] The token cache is being invalidate because project 12936b8fa28f4bebb77f3689874d24ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:45.411954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:45.412472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:45.414214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1279/2544] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:45 2022] DELETE /identity/v3/projects/12936b8fa28f4bebb77f3689874d24ac => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:45.418187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40fa6f6-ac12-49ca-bb0b-497cc7d3668a service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:45.429190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40fa6f6-ac12-49ca-bb0b-497cc7d3668a service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:45.487199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23f65b-d9df-41f3-a347-1228bc473655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:45.488190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23f65b-d9df-41f3-a347-1228bc473655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:45.488473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23f65b-d9df-41f3-a347-1228bc473655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:45.488825 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23f65b-d9df-41f3-a347-1228bc473655 admin admin] PATH_INFO: `/v3/users/4fc3af4b7eb24dcda2e713815cf33cea` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:45.490119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23f65b-d9df-41f3-a347-1228bc473655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fc3af4b7eb24dcda2e713815cf33cea)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:45.492186 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23f65b-d9df-41f3-a347-1228bc473655 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:45.549721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2b23f65b-d9df-41f3-a347-1228bc473655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fc3af4b7eb24dcda2e713815cf33cea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:45.550378 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2b23f65b-d9df-41f3-a347-1228bc473655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fc3af4b7eb24dcda2e713815cf33cea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:45.557741 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2b23f65b-d9df-41f3-a347-1228bc473655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fc3af4b7eb24dcda2e713815cf33cea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:45.569949 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2b23f65b-d9df-41f3-a347-1228bc473655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fc3af4b7eb24dcda2e713815cf33cea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:45.576843 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2b23f65b-d9df-41f3-a347-1228bc473655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fc3af4b7eb24dcda2e713815cf33cea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:45.582728 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1266/2545] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:45 2022] DELETE /identity/v3/users/4fc3af4b7eb24dcda2e713815cf33cea => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:45.857573 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:45.859052 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:45.872626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae9343-e81a-4777-8187-e60cb7b836a2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:45.931580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:45.932516 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:45.932802 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:45.933083 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e admin admin] PATH_INFO: `/v3/projects/60bf899f0ce54b388fab8652f38a542f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:45.934256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=60bf899f0ce54b388fab8652f38a542f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:45.936203 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:45.982910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60bf899f0ce54b388fab8652f38a542f', 'request_id': 'req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:45.989119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60bf899f0ce54b388fab8652f38a542f', 'request_id': 'req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:45.994683 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60bf899f0ce54b388fab8652f38a542f', 'request_id': 'req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:46.007227 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e admin admin] The token cache is being invalidate because project 60bf899f0ce54b388fab8652f38a542f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:46.007518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:46.008177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ccfac68-8c03-4ca6-9d09-4e36aaff9e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:46.009746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1280/2546] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:45 2022] DELETE /identity/v3/projects/60bf899f0ce54b388fab8652f38a542f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:46.014645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc22672-fc74-4755-99a5-b59e357e9797 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.015611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc22672-fc74-4755-99a5-b59e357e9797 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.015776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc22672-fc74-4755-99a5-b59e357e9797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.034057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fcc22672-fc74-4755-99a5-b59e357e9797 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:46.035249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fcc22672-fc74-4755-99a5-b59e357e9797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:46.083504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:46.101005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1267/2547] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:46.108636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b158b-34f9-44b0-a049-11a33f3b807e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.108929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b158b-34f9-44b0-a049-11a33f3b807e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.109216 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b158b-34f9-44b0-a049-11a33f3b807e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.127579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-917b158b-34f9-44b0-a049-11a33f3b807e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:46.128833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-917b158b-34f9-44b0-a049-11a33f3b807e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:46.136816 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1281/2548] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:46.141485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc22672-fc74-4755-99a5-b59e357e9797 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.151258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc22672-fc74-4755-99a5-b59e357e9797 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.155790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fb277-0157-4410-bf88-2b90cc381060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.156434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fb277-0157-4410-bf88-2b90cc381060 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.156719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fb277-0157-4410-bf88-2b90cc381060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.156995 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fb277-0157-4410-bf88-2b90cc381060 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.157711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fb277-0157-4410-bf88-2b90cc381060 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:46.158017 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fb277-0157-4410-bf88-2b90cc381060 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.159614 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fb277-0157-4410-bf88-2b90cc381060 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.172454 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1268/2549] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:46.176597 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b158b-34f9-44b0-a049-11a33f3b807e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.185778 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b158b-34f9-44b0-a049-11a33f3b807e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.190965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cdc9b-dd97-4e2b-8259-c1d6f64a3c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.191665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cdc9b-dd97-4e2b-8259-c1d6f64a3c74 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.191959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cdc9b-dd97-4e2b-8259-c1d6f64a3c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.192231 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cdc9b-dd97-4e2b-8259-c1d6f64a3c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.192911 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cdc9b-dd97-4e2b-8259-c1d6f64a3c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.194795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cdc9b-dd97-4e2b-8259-c1d6f64a3c74 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.220426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1282/2550] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:46 2022] POST /identity/v3/projects => generated 355 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:46.224479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fb277-0157-4410-bf88-2b90cc381060 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.234335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fb277-0157-4410-bf88-2b90cc381060 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.291487 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ab8e9-f62c-456c-893b-3a1986882294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.292176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ab8e9-f62c-456c-893b-3a1986882294 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.292462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ab8e9-f62c-456c-893b-3a1986882294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.292742 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ab8e9-f62c-456c-893b-3a1986882294 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.293432 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ab8e9-f62c-456c-893b-3a1986882294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.295343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ab8e9-f62c-456c-893b-3a1986882294 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.314871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1269/2551] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:46 2022] POST /identity/v3/users => generated 278 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:46.318900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cdc9b-dd97-4e2b-8259-c1d6f64a3c74 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.325554 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:46.330126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cdc9b-dd97-4e2b-8259-c1d6f64a3c74 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.335956 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ed819-2905-406d-b32a-6df6e53950d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.336632 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ed819-2905-406d-b32a-6df6e53950d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.337126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ed819-2905-406d-b32a-6df6e53950d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.337553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ed819-2905-406d-b32a-6df6e53950d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.338474 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ed819-2905-406d-b32a-6df6e53950d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.340122 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ed819-2905-406d-b32a-6df6e53950d2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.350689 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1283/2552] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:46.354729 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ab8e9-f62c-456c-893b-3a1986882294 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.364592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ab8e9-f62c-456c-893b-3a1986882294 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.369148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80804785-111b-4a4b-a18f-2156bab79221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.369796 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80804785-111b-4a4b-a18f-2156bab79221 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.370078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80804785-111b-4a4b-a18f-2156bab79221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.370351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80804785-111b-4a4b-a18f-2156bab79221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.371074 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80804785-111b-4a4b-a18f-2156bab79221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.372789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80804785-111b-4a4b-a18f-2156bab79221 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.382998 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1270/2553] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:46.386850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ed819-2905-406d-b32a-6df6e53950d2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.397614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ed819-2905-406d-b32a-6df6e53950d2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.402575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb920cb7-adbe-434d-8737-027e64336317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.403476 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb920cb7-adbe-434d-8737-027e64336317 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.403788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb920cb7-adbe-434d-8737-027e64336317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.404065 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb920cb7-adbe-434d-8737-027e64336317 admin admin] PATH_INFO: `/v3/projects/60bb1085d9c54b33a288d2bd848ad872/users/f1eaf25261ab48899f486f990df77d1d/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.418399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb920cb7-adbe-434d-8737-027e64336317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60bb1085d9c54b33a288d2bd848ad872, user_id=f1eaf25261ab48899f486f990df77d1d, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.421584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb920cb7-adbe-434d-8737-027e64336317 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.435386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1284/2554] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:46 2022] PUT /identity/v3/projects/60bb1085d9c54b33a288d2bd848ad872/users/f1eaf25261ab48899f486f990df77d1d/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:46.447165 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cd9c0-f404-4bd3-8887-97be83f61b59 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.447471 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cd9c0-f404-4bd3-8887-97be83f61b59 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.447766 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cd9c0-f404-4bd3-8887-97be83f61b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.473917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c22cd9c0-f404-4bd3-8887-97be83f61b59 None None] MFA Rules not processed for user `f1eaf25261ab48899f486f990df77d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:46.475116 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c22cd9c0-f404-4bd3-8887-97be83f61b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:46.537667 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1271/2555] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:46 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:46.542156 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb920cb7-adbe-434d-8737-027e64336317 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.605933 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb920cb7-adbe-434d-8737-027e64336317 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.611446 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e05d0-6c19-4ae8-92af-15b9f72d51b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.612222 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e05d0-6c19-4ae8-92af-15b9f72d51b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.612503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e05d0-6c19-4ae8-92af-15b9f72d51b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.612804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e05d0-6c19-4ae8-92af-15b9f72d51b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.613499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e05d0-6c19-4ae8-92af-15b9f72d51b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.615432 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e05d0-6c19-4ae8-92af-15b9f72d51b3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.618950 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cd9c0-f404-4bd3-8887-97be83f61b59 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.620964 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c22cd9c0-f404-4bd3-8887-97be83f61b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:46.635326 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cd9c0-f404-4bd3-8887-97be83f61b59 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.640141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eabf9e-5cfc-4101-8df7-56a932aab045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.641013 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabf9e-5cfc-4101-8df7-56a932aab045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.641297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabf9e-5cfc-4101-8df7-56a932aab045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.641578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabf9e-5cfc-4101-8df7-56a932aab045 admin admin] PATH_INFO: `/v3/users/923bcaa783814db2b09440356d8a90ff` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.642225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1285/2556] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:46 2022] POST /identity/v3/projects => generated 357 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:46.646232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eabf9e-5cfc-4101-8df7-56a932aab045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=923bcaa783814db2b09440356d8a90ff)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.646367 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e05d0-6c19-4ae8-92af-15b9f72d51b3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.648374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eabf9e-5cfc-4101-8df7-56a932aab045 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.697524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:46.699252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b6eabf9e-5cfc-4101-8df7-56a932aab045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '923bcaa783814db2b09440356d8a90ff'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:46.700001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b6eabf9e-5cfc-4101-8df7-56a932aab045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '923bcaa783814db2b09440356d8a90ff'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:46.708751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b6eabf9e-5cfc-4101-8df7-56a932aab045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '923bcaa783814db2b09440356d8a90ff'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:46.710293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e05d0-6c19-4ae8-92af-15b9f72d51b3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.715187 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1002f3e-6765-408f-a7dc-74ca166a92b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.715886 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1002f3e-6765-408f-a7dc-74ca166a92b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.716294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1002f3e-6765-408f-a7dc-74ca166a92b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.716580 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1002f3e-6765-408f-a7dc-74ca166a92b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.717285 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1002f3e-6765-408f-a7dc-74ca166a92b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.719214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1002f3e-6765-408f-a7dc-74ca166a92b9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.723002 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b6eabf9e-5cfc-4101-8df7-56a932aab045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '923bcaa783814db2b09440356d8a90ff'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:46.731139 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b6eabf9e-5cfc-4101-8df7-56a932aab045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '923bcaa783814db2b09440356d8a90ff'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:46.738060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1272/2557] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:46 2022] DELETE /identity/v3/users/923bcaa783814db2b09440356d8a90ff => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:46.740709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1286/2558] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:46 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:10:46.742563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c238b0f-5e12-49ca-987b-3aa21e742d96 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.742863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c238b0f-5e12-49ca-987b-3aa21e742d96 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.743147 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c238b0f-5e12-49ca-987b-3aa21e742d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.744386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1002f3e-6765-408f-a7dc-74ca166a92b9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.745997 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c1002f3e-6765-408f-a7dc-74ca166a92b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:46.758851 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1002f3e-6765-408f-a7dc-74ca166a92b9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.761155 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7c238b0f-5e12-49ca-987b-3aa21e742d96 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:46.762362 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7c238b0f-5e12-49ca-987b-3aa21e742d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:46.802310 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:46.818834 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb684b-858b-432a-876b-2a6575d6fcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.819504 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb684b-858b-432a-876b-2a6575d6fcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.819822 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb684b-858b-432a-876b-2a6575d6fcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.820099 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb684b-858b-432a-876b-2a6575d6fcf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.820826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb684b-858b-432a-876b-2a6575d6fcf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.821538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1273/2559] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:10:46.822623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb684b-858b-432a-876b-2a6575d6fcf8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.831233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c238b0f-5e12-49ca-987b-3aa21e742d96 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.832621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7c238b0f-5e12-49ca-987b-3aa21e742d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:46.835734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1287/2560] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:10:46.836678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb684b-858b-432a-876b-2a6575d6fcf8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.846699 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb684b-858b-432a-876b-2a6575d6fcf8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.851336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9c530-7679-4062-89c9-34294386f954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.852226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9c530-7679-4062-89c9-34294386f954 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.852518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9c530-7679-4062-89c9-34294386f954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.852792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9c530-7679-4062-89c9-34294386f954 admin admin] PATH_INFO: `/v3/projects/670314cc1cbe473a8aaffce0dfa1e34c/users/7e278f16140b42fe914a3ceeea83581e/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.868139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9c530-7679-4062-89c9-34294386f954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=670314cc1cbe473a8aaffce0dfa1e34c, user_id=7e278f16140b42fe914a3ceeea83581e, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.870811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9c530-7679-4062-89c9-34294386f954 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.886178 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1288/2561] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:46 2022] PUT /identity/v3/projects/670314cc1cbe473a8aaffce0dfa1e34c/users/7e278f16140b42fe914a3ceeea83581e/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:46.890079 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9c530-7679-4062-89c9-34294386f954 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.890447 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c238b0f-5e12-49ca-987b-3aa21e742d96 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.940027 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff89df2-8ca5-4bd9-a061-3de4fc6bc3a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.940684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff89df2-8ca5-4bd9-a061-3de4fc6bc3a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.940971 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff89df2-8ca5-4bd9-a061-3de4fc6bc3a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.941253 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff89df2-8ca5-4bd9-a061-3de4fc6bc3a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.951556 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9c530-7679-4062-89c9-34294386f954 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.956452 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39143a25-4e3f-455c-ab31-868603949d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.957403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39143a25-4e3f-455c-ab31-868603949d6d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.957690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39143a25-4e3f-455c-ab31-868603949d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.957980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39143a25-4e3f-455c-ab31-868603949d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.958707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39143a25-4e3f-455c-ab31-868603949d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.960367 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39143a25-4e3f-455c-ab31-868603949d6d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.974012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1289/2562] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:10:46.974256 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:46.975161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39143a25-4e3f-455c-ab31-868603949d6d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.986768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39143a25-4e3f-455c-ab31-868603949d6d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.991508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc0980-9221-4783-bc39-dc273093637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.992426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc0980-9221-4783-bc39-dc273093637d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.992707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc0980-9221-4783-bc39-dc273093637d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.992991 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc0980-9221-4783-bc39-dc273093637d admin admin] PATH_INFO: `/v3/projects/670314cc1cbe473a8aaffce0dfa1e34c/users/7e278f16140b42fe914a3ceeea83581e/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.994983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc0980-9221-4783-bc39-dc273093637d admin admin] RBAC: Authorizing `identity:create_grant(project_id=670314cc1cbe473a8aaffce0dfa1e34c, user_id=7e278f16140b42fe914a3ceeea83581e, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.997687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc0980-9221-4783-bc39-dc273093637d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:47.007948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1290/2563] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:46 2022] PUT /identity/v3/projects/670314cc1cbe473a8aaffce0dfa1e34c/users/7e278f16140b42fe914a3ceeea83581e/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:47.016577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff89df2-8ca5-4bd9-a061-3de4fc6bc3a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:47.016577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff89df2-8ca5-4bd9-a061-3de4fc6bc3a8 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:47.018197 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb053d3-181a-4e44-b689-fb1d2666eaf3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:47.018490 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb053d3-181a-4e44-b689-fb1d2666eaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:47.018770 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb053d3-181a-4e44-b689-fb1d2666eaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:47.019635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1274/2564] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:10:47.045176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4bb053d3-181a-4e44-b689-fb1d2666eaf3 None None] MFA Rules not processed for user `7e278f16140b42fe914a3ceeea83581e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:47.046965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4bb053d3-181a-4e44-b689-fb1d2666eaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:47.123508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1291/2565] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:47 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:10:47.134581 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff89df2-8ca5-4bd9-a061-3de4fc6bc3a8 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:47.136058 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5ff89df2-8ca5-4bd9-a061-3de4fc6bc3a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:47.199680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff89df2-8ca5-4bd9-a061-3de4fc6bc3a8 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:47.250528 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bb7b3-e7cf-456f-a488-6c5443d25cdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:47.251177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bb7b3-e7cf-456f-a488-6c5443d25cdf service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:47.251461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bb7b3-e7cf-456f-a488-6c5443d25cdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:47.251804 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bb7b3-e7cf-456f-a488-6c5443d25cdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:47.261100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3bb7b3-e7cf-456f-a488-6c5443d25cdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:47.262527 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3bb7b3-e7cf-456f-a488-6c5443d25cdf service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:47.274418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1275/2566] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:47 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:47.292435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb053d3-181a-4e44-b689-fb1d2666eaf3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:47.294019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4bb053d3-181a-4e44-b689-fb1d2666eaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:47.331626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:47.354761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bb7b3-e7cf-456f-a488-6c5443d25cdf service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:47.363480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bb7b3-e7cf-456f-a488-6c5443d25cdf service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:47.367308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c343106-e4a5-4cbf-a697-dfa77e95db99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:47.367983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c343106-e4a5-4cbf-a697-dfa77e95db99 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:47.368271 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c343106-e4a5-4cbf-a697-dfa77e95db99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:47.368578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c343106-e4a5-4cbf-a697-dfa77e95db99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:47.370370 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4c343106-e4a5-4cbf-a697-dfa77e95db99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:47.372667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb053d3-181a-4e44-b689-fb1d2666eaf3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:47.377435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:47.378294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:47.378578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:47.378992 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] PATH_INFO: `/v3/projects/5b5e7aba22dd4bf6992cb37605e50a11` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:47.380219 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b5e7aba22dd4bf6992cb37605e50a11)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:47.382139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:47.431697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b5e7aba22dd4bf6992cb37605e50a11', 'request_id': 'req-71e89d9d-85db-4129-89b1-1964fd8cc535'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.434719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c343106-e4a5-4cbf-a697-dfa77e95db99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:47.437111 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c343106-e4a5-4cbf-a697-dfa77e95db99 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:47.438707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b5e7aba22dd4bf6992cb37605e50a11', 'request_id': 'req-71e89d9d-85db-4129-89b1-1964fd8cc535'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.445331 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b5e7aba22dd4bf6992cb37605e50a11', 'request_id': 'req-71e89d9d-85db-4129-89b1-1964fd8cc535'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.450145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1276/2567] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:47 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:47.461016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] The token cache is being invalidate because project 5b5e7aba22dd4bf6992cb37605e50a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:47.461435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.462254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.463982 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1292/2568] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:47 2022] DELETE /identity/v3/projects/5b5e7aba22dd4bf6992cb37605e50a11 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:47.468525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c343106-e4a5-4cbf-a697-dfa77e95db99 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:47.479306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c343106-e4a5-4cbf-a697-dfa77e95db99 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:47.503878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:47.543791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:47.544709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:47.544999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:47.545283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] PATH_INFO: `/v3/users/e260e63c9ac2484b80b74a9052d00e44` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:47.546855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e260e63c9ac2484b80b74a9052d00e44)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:47.548587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:47.603244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e260e63c9ac2484b80b74a9052d00e44'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.603941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e260e63c9ac2484b80b74a9052d00e44'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.611180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e260e63c9ac2484b80b74a9052d00e44'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.623699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e260e63c9ac2484b80b74a9052d00e44'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.631904 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e260e63c9ac2484b80b74a9052d00e44'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.638091 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1277/2569] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:47 2022] DELETE /identity/v3/users/e260e63c9ac2484b80b74a9052d00e44 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:47.903462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:47.905097 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:47.917909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e89d9d-85db-4129-89b1-1964fd8cc535 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:47.974594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:47.975451 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:47.975768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:47.976046 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] PATH_INFO: `/v3/projects/83d0bf54d6fd4af1a1ee8038e1a8cf17` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:47.977212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] RBAC: Authorizing `identity:delete_project(project_id=83d0bf54d6fd4af1a1ee8038e1a8cf17)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:47.979368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:48.024648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83d0bf54d6fd4af1a1ee8038e1a8cf17', 'request_id': 'req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.030536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83d0bf54d6fd4af1a1ee8038e1a8cf17', 'request_id': 'req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.032473 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:48.036892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83d0bf54d6fd4af1a1ee8038e1a8cf17', 'request_id': 'req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.049290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] The token cache is being invalidate because project 83d0bf54d6fd4af1a1ee8038e1a8cf17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:48.049579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.050192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.051722 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1293/2570] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:47 2022] DELETE /identity/v3/projects/83d0bf54d6fd4af1a1ee8038e1a8cf17 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:48.055919 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:48.057322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:48.070476 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef9e08-8808-46c1-8d24-6613f3464376 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:48.128945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45343606-175d-4951-97d0-67cf60ba25b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:48.129883 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45343606-175d-4951-97d0-67cf60ba25b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.130176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45343606-175d-4951-97d0-67cf60ba25b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.130452 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45343606-175d-4951-97d0-67cf60ba25b2 admin admin] PATH_INFO: `/v3/users/79727dc36ac9424fb2a8b8afaba006e1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.131702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45343606-175d-4951-97d0-67cf60ba25b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79727dc36ac9424fb2a8b8afaba006e1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:48.133648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45343606-175d-4951-97d0-67cf60ba25b2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:48.180016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-45343606-175d-4951-97d0-67cf60ba25b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79727dc36ac9424fb2a8b8afaba006e1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.180576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-45343606-175d-4951-97d0-67cf60ba25b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79727dc36ac9424fb2a8b8afaba006e1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.182232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:48.189543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-45343606-175d-4951-97d0-67cf60ba25b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79727dc36ac9424fb2a8b8afaba006e1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.203049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-45343606-175d-4951-97d0-67cf60ba25b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79727dc36ac9424fb2a8b8afaba006e1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.209835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-45343606-175d-4951-97d0-67cf60ba25b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79727dc36ac9424fb2a8b8afaba006e1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.215811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1278/2571] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:48 2022] DELETE /identity/v3/users/79727dc36ac9424fb2a8b8afaba006e1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:48.480796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:48.482243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:48.502421 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7e0f7-2c7c-494e-ba3d-b97a9428995c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:48.558695 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:48.559555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.559877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.560153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110 admin admin] PATH_INFO: `/v3/projects/e316efb172de48a685718fc384b5d5e3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.561325 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e316efb172de48a685718fc384b5d5e3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:48.563265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:48.610896 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e316efb172de48a685718fc384b5d5e3', 'request_id': 'req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.617362 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e316efb172de48a685718fc384b5d5e3', 'request_id': 'req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.623198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e316efb172de48a685718fc384b5d5e3', 'request_id': 'req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.637667 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110 admin admin] The token cache is being invalidate because project e316efb172de48a685718fc384b5d5e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:48.637956 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.638603 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dfdbbed6-f1fe-415a-9828-c63b9f4fd110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.640934 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1294/2572] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:48 2022] DELETE /identity/v3/projects/e316efb172de48a685718fc384b5d5e3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:48.645984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce28c4c-298d-42ad-b41e-01305c170235 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.646269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce28c4c-298d-42ad-b41e-01305c170235 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.646543 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce28c4c-298d-42ad-b41e-01305c170235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.666039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3ce28c4c-298d-42ad-b41e-01305c170235 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:48.667320 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3ce28c4c-298d-42ad-b41e-01305c170235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:48.746950 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1279/2573] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:48.748463 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7abddb-2eb0-4cbc-8947-2b2c3d07c810 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.748865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7abddb-2eb0-4cbc-8947-2b2c3d07c810 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.749343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7abddb-2eb0-4cbc-8947-2b2c3d07c810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.774181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5c7abddb-2eb0-4cbc-8947-2b2c3d07c810 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:48.775394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5c7abddb-2eb0-4cbc-8947-2b2c3d07c810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:48.784093 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1295/2574] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:48.788823 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce28c4c-298d-42ad-b41e-01305c170235 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:48.799802 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce28c4c-298d-42ad-b41e-01305c170235 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:48.804297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4c46b-c07b-40b4-a93c-aa298f5f4f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:48.804945 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4c46b-c07b-40b4-a93c-aa298f5f4f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.805579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4c46b-c07b-40b4-a93c-aa298f5f4f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.805857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4c46b-c07b-40b4-a93c-aa298f5f4f67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.806744 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4c46b-c07b-40b4-a93c-aa298f5f4f67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:48.807054 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4c46b-c07b-40b4-a93c-aa298f5f4f67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:48.808897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4c46b-c07b-40b4-a93c-aa298f5f4f67 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:48.820636 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1280/2575] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:48.824790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7abddb-2eb0-4cbc-8947-2b2c3d07c810 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:48.834430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7abddb-2eb0-4cbc-8947-2b2c3d07c810 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:48.839062 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b53bc4-ae20-4c65-ad02-ba1609583fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:48.839741 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b53bc4-ae20-4c65-ad02-ba1609583fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.840029 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b53bc4-ae20-4c65-ad02-ba1609583fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.840308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b53bc4-ae20-4c65-ad02-ba1609583fe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.841011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b53bc4-ae20-4c65-ad02-ba1609583fe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:48.843621 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b53bc4-ae20-4c65-ad02-ba1609583fe4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:48.868351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1296/2576] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:48 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:48.872313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4c46b-c07b-40b4-a93c-aa298f5f4f67 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:48.882175 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4c46b-c07b-40b4-a93c-aa298f5f4f67 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:48.936857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81751122-6faa-4316-8c11-d28249934d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:48.937517 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81751122-6faa-4316-8c11-d28249934d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.937801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81751122-6faa-4316-8c11-d28249934d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.938078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81751122-6faa-4316-8c11-d28249934d73 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.938755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81751122-6faa-4316-8c11-d28249934d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:48.940722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81751122-6faa-4316-8c11-d28249934d73 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:48.960722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1281/2577] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:48 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:48.964959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b53bc4-ae20-4c65-ad02-ba1609583fe4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:48.975874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b53bc4-ae20-4c65-ad02-ba1609583fe4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:48.982027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbc7c2-7ad2-4c4c-884d-ab41e0bbe3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:48.982901 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbc7c2-7ad2-4c4c-884d-ab41e0bbe3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.983288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbc7c2-7ad2-4c4c-884d-ab41e0bbe3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.983578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbc7c2-7ad2-4c4c-884d-ab41e0bbe3c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.984426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbc7c2-7ad2-4c4c-884d-ab41e0bbe3c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:48.986298 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbc7c2-7ad2-4c4c-884d-ab41e0bbe3c7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:48.997796 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1297/2578] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:49.002225 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81751122-6faa-4316-8c11-d28249934d73 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:49.012098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81751122-6faa-4316-8c11-d28249934d73 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:49.017319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5da601-f4b7-4607-bdab-08a6a4a8a7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:49.018219 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5da601-f4b7-4607-bdab-08a6a4a8a7df admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.018507 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5da601-f4b7-4607-bdab-08a6a4a8a7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.018780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5da601-f4b7-4607-bdab-08a6a4a8a7df admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.019500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5da601-f4b7-4607-bdab-08a6a4a8a7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:49.021181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5da601-f4b7-4607-bdab-08a6a4a8a7df admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:49.030978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1282/2579] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:49.035158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbc7c2-7ad2-4c4c-884d-ab41e0bbe3c7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:49.044963 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbc7c2-7ad2-4c4c-884d-ab41e0bbe3c7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:49.049548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7de3c7-235c-457a-851a-aab50a278f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:49.050407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7de3c7-235c-457a-851a-aab50a278f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.050684 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7de3c7-235c-457a-851a-aab50a278f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.050956 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7de3c7-235c-457a-851a-aab50a278f83 admin admin] PATH_INFO: `/v3/projects/c8040a7c927f4d4ebff4964bc1b63ad7/users/57cabfbd1c494f62b291be319781b071/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.065503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7de3c7-235c-457a-851a-aab50a278f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8040a7c927f4d4ebff4964bc1b63ad7, user_id=57cabfbd1c494f62b291be319781b071, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:49.068177 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7de3c7-235c-457a-851a-aab50a278f83 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:49.081363 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1298/2580] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:49 2022] PUT /identity/v3/projects/c8040a7c927f4d4ebff4964bc1b63ad7/users/57cabfbd1c494f62b291be319781b071/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:49.086366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e7189-fa4e-4875-81d0-776eeccaccd4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.086817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e7189-fa4e-4875-81d0-776eeccaccd4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.086990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e7189-fa4e-4875-81d0-776eeccaccd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.104826 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-eb4e7189-fa4e-4875-81d0-776eeccaccd4 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:49.106012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eb4e7189-fa4e-4875-81d0-776eeccaccd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:49.131644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:49.166648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1283/2581] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:10:49.176411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7de3c7-235c-457a-851a-aab50a278f83 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:49.178078 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1c7de3c7-235c-457a-851a-aab50a278f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:49.191779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:49.236294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7de3c7-235c-457a-851a-aab50a278f83 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:49.239836 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2a6c3-0e17-41ad-9c1b-32faf41ef923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:49.240475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2a6c3-0e17-41ad-9c1b-32faf41ef923 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.240754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2a6c3-0e17-41ad-9c1b-32faf41ef923 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.241026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2a6c3-0e17-41ad-9c1b-32faf41ef923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.250360 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2a6c3-0e17-41ad-9c1b-32faf41ef923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:49.251814 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2a6c3-0e17-41ad-9c1b-32faf41ef923 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:49.261674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1299/2582] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:53.625492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e7189-fa4e-4875-81d0-776eeccaccd4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:53.655094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eb4e7189-fa4e-4875-81d0-776eeccaccd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:53.678393 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2300fe-f4fc-4cd6-8f8e-288f16155efb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:53.678958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2300fe-f4fc-4cd6-8f8e-288f16155efb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:53.678958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2300fe-f4fc-4cd6-8f8e-288f16155efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:53.718352 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ee2300fe-f4fc-4cd6-8f8e-288f16155efb None None] MFA Rules not processed for user `57cabfbd1c494f62b291be319781b071`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:53.720178 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ee2300fe-f4fc-4cd6-8f8e-288f16155efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:53.766322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e7189-fa4e-4875-81d0-776eeccaccd4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:53.773691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e99be0-d5c6-46d8-9a2f-71d6482b6358 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:53.774633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e99be0-d5c6-46d8-9a2f-71d6482b6358 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:53.775395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e99be0-d5c6-46d8-9a2f-71d6482b6358 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:53.775907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e99be0-d5c6-46d8-9a2f-71d6482b6358 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:53.779150 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a4e99be0-d5c6-46d8-9a2f-71d6482b6358 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:53.793249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e99be0-d5c6-46d8-9a2f-71d6482b6358 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:53.794547 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1300/2583] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:53.795309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e99be0-d5c6-46d8-9a2f-71d6482b6358 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:53.799143 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2300fe-f4fc-4cd6-8f8e-288f16155efb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:53.806613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1284/2584] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:53.810191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2300fe-f4fc-4cd6-8f8e-288f16155efb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:53.816518 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2589838-dcd5-4258-bf79-85203649c9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:53.817300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2589838-dcd5-4258-bf79-85203649c9c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:53.817684 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2589838-dcd5-4258-bf79-85203649c9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:53.818083 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2589838-dcd5-4258-bf79-85203649c9c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:53.818891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2589838-dcd5-4258-bf79-85203649c9c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:53.821245 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2589838-dcd5-4258-bf79-85203649c9c7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:53.851045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1301/2585] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:53 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:53.855573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e99be0-d5c6-46d8-9a2f-71d6482b6358 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:53.887794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e99be0-d5c6-46d8-9a2f-71d6482b6358 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:53.979165 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ad67-fc93-4eeb-bddc-030657b1760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:53.980104 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ad67-fc93-4eeb-bddc-030657b1760c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:53.980409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ad67-fc93-4eeb-bddc-030657b1760c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:53.980763 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ad67-fc93-4eeb-bddc-030657b1760c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:53.981509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81ad67-fc93-4eeb-bddc-030657b1760c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:53.984714 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81ad67-fc93-4eeb-bddc-030657b1760c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.010321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1285/2586] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:53 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:54.014345 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2589838-dcd5-4258-bf79-85203649c9c7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.027952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2589838-dcd5-4258-bf79-85203649c9c7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.039871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b300e-f427-4382-9c37-6bd7ee51f491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.040567 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b300e-f427-4382-9c37-6bd7ee51f491 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.040916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b300e-f427-4382-9c37-6bd7ee51f491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.041225 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b300e-f427-4382-9c37-6bd7ee51f491 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.042089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b300e-f427-4382-9c37-6bd7ee51f491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.043749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b300e-f427-4382-9c37-6bd7ee51f491 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.062661 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1302/2587] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:54.067097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ad67-fc93-4eeb-bddc-030657b1760c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.079110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ad67-fc93-4eeb-bddc-030657b1760c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.086199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1e7a1-b7a4-4fa1-a9bb-9121b26b7abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.087104 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1e7a1-b7a4-4fa1-a9bb-9121b26b7abe admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.087370 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1e7a1-b7a4-4fa1-a9bb-9121b26b7abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.087754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1e7a1-b7a4-4fa1-a9bb-9121b26b7abe admin admin] PATH_INFO: `/v3/projects/f4fc5691fa8047b78be8470190280941/users/6dcaa4697f8945c8a1395dea0cd3f274/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.110035 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1e7a1-b7a4-4fa1-a9bb-9121b26b7abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4fc5691fa8047b78be8470190280941, user_id=6dcaa4697f8945c8a1395dea0cd3f274, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.113537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1e7a1-b7a4-4fa1-a9bb-9121b26b7abe admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.141670 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1286/2588] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:54 2022] PUT /identity/v3/projects/f4fc5691fa8047b78be8470190280941/users/6dcaa4697f8945c8a1395dea0cd3f274/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:54.145652 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b300e-f427-4382-9c37-6bd7ee51f491 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.164120 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b300e-f427-4382-9c37-6bd7ee51f491 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.277892 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f199f-3ca0-4f21-8fcf-8314cda98cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.278620 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f199f-3ca0-4f21-8fcf-8314cda98cec admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.278971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f199f-3ca0-4f21-8fcf-8314cda98cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.279321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f199f-3ca0-4f21-8fcf-8314cda98cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.280170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f199f-3ca0-4f21-8fcf-8314cda98cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.282733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f199f-3ca0-4f21-8fcf-8314cda98cec admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.295276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1303/2589] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:54.299959 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1e7a1-b7a4-4fa1-a9bb-9121b26b7abe admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.314694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1e7a1-b7a4-4fa1-a9bb-9121b26b7abe admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.322850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bada86d-4f07-4702-a169-a6370e51e9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.323642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bada86d-4f07-4702-a169-a6370e51e9c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.324083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bada86d-4f07-4702-a169-a6370e51e9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.324609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bada86d-4f07-4702-a169-a6370e51e9c2 admin admin] PATH_INFO: `/v3/projects/f4fc5691fa8047b78be8470190280941/users/6dcaa4697f8945c8a1395dea0cd3f274/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.336040 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bada86d-4f07-4702-a169-a6370e51e9c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4fc5691fa8047b78be8470190280941, user_id=6dcaa4697f8945c8a1395dea0cd3f274, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.336732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bada86d-4f07-4702-a169-a6370e51e9c2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.379010 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1287/2590] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:54 2022] PUT /identity/v3/projects/f4fc5691fa8047b78be8470190280941/users/6dcaa4697f8945c8a1395dea0cd3f274/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:59.326562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f199f-3ca0-4f21-8fcf-8314cda98cec admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:59.337646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:59.442664 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f199f-3ca0-4f21-8fcf-8314cda98cec admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:59.524300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafd2da-2569-43c1-9e91-5ac59603279d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:59.525891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafd2da-2569-43c1-9e91-5ac59603279d service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:59.526183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafd2da-2569-43c1-9e91-5ac59603279d service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:59.527365 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafd2da-2569-43c1-9e91-5ac59603279d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:59.529908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8bafd2da-2569-43c1-9e91-5ac59603279d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:59.639809 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafd2da-2569-43c1-9e91-5ac59603279d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:59.644261 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafd2da-2569-43c1-9e91-5ac59603279d service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:59.656972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1304/2591] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:59.792308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1446d-e298-487a-9180-7a5b822fd4b9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:59.793503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1446d-e298-487a-9180-7a5b822fd4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:59.794011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1446d-e298-487a-9180-7a5b822fd4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:59.831400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:59.836966 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-01a1446d-e298-487a-9180-7a5b822fd4b9 None None] MFA Rules not processed for user `6dcaa4697f8945c8a1395dea0cd3f274`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:59.839732 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-01a1446d-e298-487a-9180-7a5b822fd4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:59.932090 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1288/2592] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:10:59 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:59.945992 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafd2da-2569-43c1-9e91-5ac59603279d service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:59.947501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8bafd2da-2569-43c1-9e91-5ac59603279d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:00.039242 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafd2da-2569-43c1-9e91-5ac59603279d service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:00.044881 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e800de-0a7f-4b24-b510-f9cfd41a81a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:00.045540 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e800de-0a7f-4b24-b510-f9cfd41a81a6 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:00.045824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e800de-0a7f-4b24-b510-f9cfd41a81a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:00.046631 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e800de-0a7f-4b24-b510-f9cfd41a81a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:00.055606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:00.142916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e800de-0a7f-4b24-b510-f9cfd41a81a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:00.144875 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e800de-0a7f-4b24-b510-f9cfd41a81a6 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:00.155191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1305/2593] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:01.808693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ade5b7-9e71-4e05-8ef3-cd51cbe7d014 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:01.809591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ade5b7-9e71-4e05-8ef3-cd51cbe7d014 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:01.809880 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ade5b7-9e71-4e05-8ef3-cd51cbe7d014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:01.830520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-52ade5b7-9e71-4e05-8ef3-cd51cbe7d014 None None] MFA Rules not processed for user `57cabfbd1c494f62b291be319781b071`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:01.831790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-52ade5b7-9e71-4e05-8ef3-cd51cbe7d014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:01.840239 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1289/2594] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:03.407557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e800de-0a7f-4b24-b510-f9cfd41a81a6 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:03.410272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-34e800de-0a7f-4b24-b510-f9cfd41a81a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:03.433072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e800de-0a7f-4b24-b510-f9cfd41a81a6 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:03.440948 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:03.441906 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:03.442308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:03.442587 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] PATH_INFO: `/v3/users/94bcc16975124f728459d3c28b97f1a8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:03.444498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] RBAC: Authorizing `identity:delete_user(user_id=94bcc16975124f728459d3c28b97f1a8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:03.446519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:03.528248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94bcc16975124f728459d3c28b97f1a8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:03.529100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94bcc16975124f728459d3c28b97f1a8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:03.529751 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94bcc16975124f728459d3c28b97f1a8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:03.543471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94bcc16975124f728459d3c28b97f1a8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:03.557637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94bcc16975124f728459d3c28b97f1a8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:03.580228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1306/2595] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:03 2022] DELETE /identity/v3/users/94bcc16975124f728459d3c28b97f1a8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:03.839369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ade5b7-9e71-4e05-8ef3-cd51cbe7d014 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:03.861292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ade5b7-9e71-4e05-8ef3-cd51cbe7d014 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:03.952443 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:03.954019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:03.959366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:03.960274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:03.960751 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:03.961191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4 admin admin] PATH_INFO: `/v3/projects/172a9e51095e47318b27881dab868355` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:03.962782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=172a9e51095e47318b27881dab868355)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:03.964986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:03.969395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48565f-8e6d-46be-a917-03ae5e79d67c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:03.976068 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078c44b-580d-4513-b9bc-e43c89b17716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:03.976958 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078c44b-580d-4513-b9bc-e43c89b17716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:03.977245 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078c44b-580d-4513-b9bc-e43c89b17716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:03.977529 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078c44b-580d-4513-b9bc-e43c89b17716 admin admin] PATH_INFO: `/v3/users/f1eaf25261ab48899f486f990df77d1d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:03.979819 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078c44b-580d-4513-b9bc-e43c89b17716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1eaf25261ab48899f486f990df77d1d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:03.982208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078c44b-580d-4513-b9bc-e43c89b17716 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:04.037103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172a9e51095e47318b27881dab868355', 'request_id': 'req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:04.049319 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172a9e51095e47318b27881dab868355', 'request_id': 'req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:04.055257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1078c44b-580d-4513-b9bc-e43c89b17716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1eaf25261ab48899f486f990df77d1d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:04.056278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1078c44b-580d-4513-b9bc-e43c89b17716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1eaf25261ab48899f486f990df77d1d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:04.056969 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1078c44b-580d-4513-b9bc-e43c89b17716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1eaf25261ab48899f486f990df77d1d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:04.071066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172a9e51095e47318b27881dab868355', 'request_id': 'req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:04.083886 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1078c44b-580d-4513-b9bc-e43c89b17716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1eaf25261ab48899f486f990df77d1d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:04.085135 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4 admin admin] The token cache is being invalidate because project 172a9e51095e47318b27881dab868355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:04.085515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:04.086248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5629c6ec-9034-4a61-9d9c-eb1c715bf9e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:04.088081 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1290/2596] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:03 2022] DELETE /identity/v3/projects/172a9e51095e47318b27881dab868355 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:04.094061 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b59feb-c5f6-4e5a-9a4e-96ca4a1779df None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:04.094353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b59feb-c5f6-4e5a-9a4e-96ca4a1779df None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:04.094629 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b59feb-c5f6-4e5a-9a4e-96ca4a1779df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:04.097224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1078c44b-580d-4513-b9bc-e43c89b17716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1eaf25261ab48899f486f990df77d1d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:04.105718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:04.127422 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1307/2597] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:03 2022] DELETE /identity/v3/users/f1eaf25261ab48899f486f990df77d1d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:04.129072 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93479a-a288-4353-965d-9ecea62746f2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:04.129215 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-34b59feb-c5f6-4e5a-9a4e-96ca4a1779df None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:04.129465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93479a-a288-4353-965d-9ecea62746f2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:04.129749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93479a-a288-4353-965d-9ecea62746f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:04.130769 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-34b59feb-c5f6-4e5a-9a4e-96ca4a1779df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:04.149257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2d93479a-a288-4353-965d-9ecea62746f2 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:04.151123 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:04.152125 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2d93479a-a288-4353-965d-9ecea62746f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:04.207042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1291/2598] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:04.216085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1914f8b-c8d2-4acf-8246-f6ab1ae55cbd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:04.216383 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1914f8b-c8d2-4acf-8246-f6ab1ae55cbd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:04.216667 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1914f8b-c8d2-4acf-8246-f6ab1ae55cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:04.238063 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b1914f8b-c8d2-4acf-8246-f6ab1ae55cbd None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:04.239379 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b1914f8b-c8d2-4acf-8246-f6ab1ae55cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:04.251410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1292/2599] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:04.257264 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1914f8b-c8d2-4acf-8246-f6ab1ae55cbd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:04.261694 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1308/2600] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:04.271884 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93479a-a288-4353-965d-9ecea62746f2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:04.273498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2d93479a-a288-4353-965d-9ecea62746f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:04.275991 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1914f8b-c8d2-4acf-8246-f6ab1ae55cbd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:04.282203 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968ec88-5aa0-467d-b53e-62fb668cf6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:04.283020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968ec88-5aa0-467d-b53e-62fb668cf6fd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:04.283457 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968ec88-5aa0-467d-b53e-62fb668cf6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:04.283792 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968ec88-5aa0-467d-b53e-62fb668cf6fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:04.284693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968ec88-5aa0-467d-b53e-62fb668cf6fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:04.285008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968ec88-5aa0-467d-b53e-62fb668cf6fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:04.287056 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968ec88-5aa0-467d-b53e-62fb668cf6fd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:04.302078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1293/2601] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:11:04.307293 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968ec88-5aa0-467d-b53e-62fb668cf6fd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:04.319046 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968ec88-5aa0-467d-b53e-62fb668cf6fd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:04.325608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88480300-eac1-4525-b2cd-5842cb04968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:04.326223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88480300-eac1-4525-b2cd-5842cb04968c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:04.326518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88480300-eac1-4525-b2cd-5842cb04968c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:04.326791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88480300-eac1-4525-b2cd-5842cb04968c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:04.327539 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88480300-eac1-4525-b2cd-5842cb04968c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:04.329758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88480300-eac1-4525-b2cd-5842cb04968c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:04.343551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93479a-a288-4353-965d-9ecea62746f2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:04.362623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b6794-ed30-4b8f-93bf-c8c9957e9d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:04.362623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b6794-ed30-4b8f-93bf-c8c9957e9d94 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:04.362623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b6794-ed30-4b8f-93bf-c8c9957e9d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:04.364926 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b6794-ed30-4b8f-93bf-c8c9957e9d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:04.384838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1294/2602] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:04 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:04.385834 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88480300-eac1-4525-b2cd-5842cb04968c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:04.398686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b6794-ed30-4b8f-93bf-c8c9957e9d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:04.402426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b6794-ed30-4b8f-93bf-c8c9957e9d94 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:04.414858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1309/2603] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:04.506343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88480300-eac1-4525-b2cd-5842cb04968c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:04.538512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39216e-a4d7-49d3-94e4-11336bcea2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:04.539767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39216e-a4d7-49d3-94e4-11336bcea2d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:04.540362 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39216e-a4d7-49d3-94e4-11336bcea2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:04.540838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39216e-a4d7-49d3-94e4-11336bcea2d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:04.541950 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39216e-a4d7-49d3-94e4-11336bcea2d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:04.544529 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39216e-a4d7-49d3-94e4-11336bcea2d0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:04.575404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1295/2604] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:04 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:04.580710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b6794-ed30-4b8f-93bf-c8c9957e9d94 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:04.591400 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b6794-ed30-4b8f-93bf-c8c9957e9d94 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:04.597563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced37642-8cbd-4310-bd4f-1d6c493be576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:04.600801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced37642-8cbd-4310-bd4f-1d6c493be576 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:04.600801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced37642-8cbd-4310-bd4f-1d6c493be576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:04.600801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced37642-8cbd-4310-bd4f-1d6c493be576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:04.600801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced37642-8cbd-4310-bd4f-1d6c493be576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:04.602027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced37642-8cbd-4310-bd4f-1d6c493be576 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:04.619067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39216e-a4d7-49d3-94e4-11336bcea2d0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:04.620017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1310/2605] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:04.636886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39216e-a4d7-49d3-94e4-11336bcea2d0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:04.642781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1ac55-69d1-4999-89e5-f966a11732de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:04.644810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1ac55-69d1-4999-89e5-f966a11732de admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:04.645107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1ac55-69d1-4999-89e5-f966a11732de admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:04.645462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1ac55-69d1-4999-89e5-f966a11732de admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:04.646297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1ac55-69d1-4999-89e5-f966a11732de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:04.647946 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1ac55-69d1-4999-89e5-f966a11732de admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:04.661782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1296/2606] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:04.666186 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced37642-8cbd-4310-bd4f-1d6c493be576 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:04.692402 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced37642-8cbd-4310-bd4f-1d6c493be576 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:04.697448 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9e4c3-b055-4860-9922-46bc9275eb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:04.698337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9e4c3-b055-4860-9922-46bc9275eb1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:04.698623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9e4c3-b055-4860-9922-46bc9275eb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:04.698903 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9e4c3-b055-4860-9922-46bc9275eb1f admin admin] PATH_INFO: `/v3/projects/2c2f11f9f5cc4b04bb72e54fc5de2e0d/users/566687399b5544de9fc5e11452a5a13b/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:04.717234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9e4c3-b055-4860-9922-46bc9275eb1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f11f9f5cc4b04bb72e54fc5de2e0d, user_id=566687399b5544de9fc5e11452a5a13b, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:04.721411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9e4c3-b055-4860-9922-46bc9275eb1f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:04.736819 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1311/2607] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:04 2022] PUT /identity/v3/projects/2c2f11f9f5cc4b04bb72e54fc5de2e0d/users/566687399b5544de9fc5e11452a5a13b/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:04.747187 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc196e9-5a48-4ccc-8311-3383362aa86b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:04.747477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc196e9-5a48-4ccc-8311-3383362aa86b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:04.747780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc196e9-5a48-4ccc-8311-3383362aa86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:04.772963 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9e4c3-b055-4860-9922-46bc9275eb1f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:04.774721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7ce9e4c3-b055-4860-9922-46bc9275eb1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:04.810931 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fbc196e9-5a48-4ccc-8311-3383362aa86b None None] MFA Rules not processed for user `566687399b5544de9fc5e11452a5a13b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:04.814537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fbc196e9-5a48-4ccc-8311-3383362aa86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:04.902801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:04.918168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9e4c3-b055-4860-9922-46bc9275eb1f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:04.926951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:04.927941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:04.928434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:04.928701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1297/2608] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:04.928825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] PATH_INFO: `/v3/projects/60bb1085d9c54b33a288d2bd848ad872` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:04.930294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60bb1085d9c54b33a288d2bd848ad872)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:04.932379 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:04.935742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:04.942107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc196e9-5a48-4ccc-8311-3383362aa86b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:04.943546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fbc196e9-5a48-4ccc-8311-3383362aa86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:05.015567 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60bb1085d9c54b33a288d2bd848ad872', 'request_id': 'req-07524948-c538-4649-a0e7-746faf6ef161'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.024967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60bb1085d9c54b33a288d2bd848ad872', 'request_id': 'req-07524948-c538-4649-a0e7-746faf6ef161'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.035798 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60bb1085d9c54b33a288d2bd848ad872', 'request_id': 'req-07524948-c538-4649-a0e7-746faf6ef161'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.052395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc196e9-5a48-4ccc-8311-3383362aa86b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:05.054206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] The token cache is being invalidate because project 60bb1085d9c54b33a288d2bd848ad872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:05.054515 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.055213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.057019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1312/2609] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:04 2022] DELETE /identity/v3/projects/60bb1085d9c54b33a288d2bd848ad872 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:05.063137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:05.064760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:05.159145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541ad61-97c3-43db-bf40-bf8fee53e97d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:05.160029 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541ad61-97c3-43db-bf40-bf8fee53e97d service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:05.160333 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541ad61-97c3-43db-bf40-bf8fee53e97d service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:05.160651 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541ad61-97c3-43db-bf40-bf8fee53e97d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:05.162917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e541ad61-97c3-43db-bf40-bf8fee53e97d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:05.169022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07524948-c538-4649-a0e7-746faf6ef161 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:05.175749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5a761-85bc-4ebf-ad13-8deb6f71a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:05.176649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5a761-85bc-4ebf-ad13-8deb6f71a222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:05.176937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5a761-85bc-4ebf-ad13-8deb6f71a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:05.177219 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5a761-85bc-4ebf-ad13-8deb6f71a222 admin admin] PATH_INFO: `/v3/users/7e278f16140b42fe914a3ceeea83581e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:05.178731 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5a761-85bc-4ebf-ad13-8deb6f71a222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e278f16140b42fe914a3ceeea83581e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:05.181809 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5a761-85bc-4ebf-ad13-8deb6f71a222 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:05.239614 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541ad61-97c3-43db-bf40-bf8fee53e97d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:05.241274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541ad61-97c3-43db-bf40-bf8fee53e97d service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:05.259395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3ac5a761-85bc-4ebf-ad13-8deb6f71a222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e278f16140b42fe914a3ceeea83581e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.260087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1298/2610] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:05.260383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3ac5a761-85bc-4ebf-ad13-8deb6f71a222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e278f16140b42fe914a3ceeea83581e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.261114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3ac5a761-85bc-4ebf-ad13-8deb6f71a222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e278f16140b42fe914a3ceeea83581e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.281457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3ac5a761-85bc-4ebf-ad13-8deb6f71a222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e278f16140b42fe914a3ceeea83581e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.292806 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3ac5a761-85bc-4ebf-ad13-8deb6f71a222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e278f16140b42fe914a3ceeea83581e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.316706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1313/2611] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:05 2022] DELETE /identity/v3/users/7e278f16140b42fe914a3ceeea83581e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:05.662243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541ad61-97c3-43db-bf40-bf8fee53e97d service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:05.677353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541ad61-97c3-43db-bf40-bf8fee53e97d service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:05.717083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:05.761579 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d4285-3012-4e6f-9652-f953847c3f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:05.762465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d4285-3012-4e6f-9652-f953847c3f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:05.762709 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d4285-3012-4e6f-9652-f953847c3f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:05.762987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d4285-3012-4e6f-9652-f953847c3f04 admin admin] PATH_INFO: `/v3/projects/670314cc1cbe473a8aaffce0dfa1e34c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:05.764200 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d4285-3012-4e6f-9652-f953847c3f04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=670314cc1cbe473a8aaffce0dfa1e34c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:05.766132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d4285-3012-4e6f-9652-f953847c3f04 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:05.814953 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1a4d4285-3012-4e6f-9652-f953847c3f04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670314cc1cbe473a8aaffce0dfa1e34c', 'request_id': 'req-1a4d4285-3012-4e6f-9652-f953847c3f04'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.823210 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1a4d4285-3012-4e6f-9652-f953847c3f04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670314cc1cbe473a8aaffce0dfa1e34c', 'request_id': 'req-1a4d4285-3012-4e6f-9652-f953847c3f04'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.830069 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1a4d4285-3012-4e6f-9652-f953847c3f04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670314cc1cbe473a8aaffce0dfa1e34c', 'request_id': 'req-1a4d4285-3012-4e6f-9652-f953847c3f04'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.841351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1a4d4285-3012-4e6f-9652-f953847c3f04 admin admin] The token cache is being invalidate because project 670314cc1cbe473a8aaffce0dfa1e34c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:05.841642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1a4d4285-3012-4e6f-9652-f953847c3f04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.842296 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1a4d4285-3012-4e6f-9652-f953847c3f04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:05.844659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1299/2612] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:05 2022] DELETE /identity/v3/projects/670314cc1cbe473a8aaffce0dfa1e34c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:05.849437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98602479-822e-4efe-b4aa-f480b41ccdef None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:05.849910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98602479-822e-4efe-b4aa-f480b41ccdef None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:05.850033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98602479-822e-4efe-b4aa-f480b41ccdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:05.870176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-98602479-822e-4efe-b4aa-f480b41ccdef None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:05.871323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-98602479-822e-4efe-b4aa-f480b41ccdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:05.877450 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:05.934160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1314/2613] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:05.942803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ab09b-8073-44b6-bf5c-d27e9846ce69 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:05.943270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ab09b-8073-44b6-bf5c-d27e9846ce69 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:05.943369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ab09b-8073-44b6-bf5c-d27e9846ce69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:05.966782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-745ab09b-8073-44b6-bf5c-d27e9846ce69 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:05.968107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-745ab09b-8073-44b6-bf5c-d27e9846ce69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:05.976676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1300/2614] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:05.981200 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98602479-822e-4efe-b4aa-f480b41ccdef None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:05.993300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98602479-822e-4efe-b4aa-f480b41ccdef None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:05.997940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fbeb8b-0196-46a3-bd02-64578d98d3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:05.998604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbeb8b-0196-46a3-bd02-64578d98d3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:05.998888 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbeb8b-0196-46a3-bd02-64578d98d3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:05.999169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbeb8b-0196-46a3-bd02-64578d98d3f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:05.999929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fbeb8b-0196-46a3-bd02-64578d98d3f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:06.000247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fbeb8b-0196-46a3-bd02-64578d98d3f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.001857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fbeb8b-0196-46a3-bd02-64578d98d3f4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.013256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1315/2615] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:06.018120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ab09b-8073-44b6-bf5c-d27e9846ce69 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.027563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ab09b-8073-44b6-bf5c-d27e9846ce69 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.032958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fdf6e-c74b-42ab-a4a4-81fd1d467732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.033660 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fdf6e-c74b-42ab-a4a4-81fd1d467732 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.033934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fdf6e-c74b-42ab-a4a4-81fd1d467732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.034206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fdf6e-c74b-42ab-a4a4-81fd1d467732 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.034890 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fdf6e-c74b-42ab-a4a4-81fd1d467732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.036806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fdf6e-c74b-42ab-a4a4-81fd1d467732 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.061033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1301/2616] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:06 2022] POST /identity/v3/projects => generated 387 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:06.065054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fbeb8b-0196-46a3-bd02-64578d98d3f4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.076491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fbeb8b-0196-46a3-bd02-64578d98d3f4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.136311 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7903800-d158-45db-afdd-08b4cc89f503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.136915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7903800-d158-45db-afdd-08b4cc89f503 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.137206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7903800-d158-45db-afdd-08b4cc89f503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.137486 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7903800-d158-45db-afdd-08b4cc89f503 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.138173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7903800-d158-45db-afdd-08b4cc89f503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.140617 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7903800-d158-45db-afdd-08b4cc89f503 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.161136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1316/2617] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:06 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:06.165352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fdf6e-c74b-42ab-a4a4-81fd1d467732 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.175776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fdf6e-c74b-42ab-a4a4-81fd1d467732 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.180472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8eddcb-61bb-4ed3-a14a-63cd393906bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.181129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8eddcb-61bb-4ed3-a14a-63cd393906bf admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.181411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8eddcb-61bb-4ed3-a14a-63cd393906bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.181689 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8eddcb-61bb-4ed3-a14a-63cd393906bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.182594 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8eddcb-61bb-4ed3-a14a-63cd393906bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.184473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8eddcb-61bb-4ed3-a14a-63cd393906bf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.197803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1302/2618] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:06.200152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7903800-d158-45db-afdd-08b4cc89f503 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.211895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7903800-d158-45db-afdd-08b4cc89f503 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.216504 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f084ea-12c7-4ce6-b5c0-7ecaa9662602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.217157 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f084ea-12c7-4ce6-b5c0-7ecaa9662602 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.217438 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f084ea-12c7-4ce6-b5c0-7ecaa9662602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.217711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f084ea-12c7-4ce6-b5c0-7ecaa9662602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.219088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f084ea-12c7-4ce6-b5c0-7ecaa9662602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.220690 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f084ea-12c7-4ce6-b5c0-7ecaa9662602 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.230381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1317/2619] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:06.234790 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8eddcb-61bb-4ed3-a14a-63cd393906bf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.244743 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8eddcb-61bb-4ed3-a14a-63cd393906bf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.249900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d45e0-dbca-4584-bf11-e4c1a32fb5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.250761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d45e0-dbca-4584-bf11-e4c1a32fb5c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.251043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d45e0-dbca-4584-bf11-e4c1a32fb5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.251318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d45e0-dbca-4584-bf11-e4c1a32fb5c0 admin admin] PATH_INFO: `/v3/projects/64572285cac84b07b702cebf00d1722f/users/5069b527255c458ead325892f641261f/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.266113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d45e0-dbca-4584-bf11-e4c1a32fb5c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64572285cac84b07b702cebf00d1722f, user_id=5069b527255c458ead325892f641261f, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.268789 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d45e0-dbca-4584-bf11-e4c1a32fb5c0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.283435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1303/2620] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:06 2022] PUT /identity/v3/projects/64572285cac84b07b702cebf00d1722f/users/5069b527255c458ead325892f641261f/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:06.293435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06827432-9267-47a3-a969-6f7da17bcb5d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.293747 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06827432-9267-47a3-a969-6f7da17bcb5d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.294012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06827432-9267-47a3-a969-6f7da17bcb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.319960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-06827432-9267-47a3-a969-6f7da17bcb5d None None] MFA Rules not processed for user `5069b527255c458ead325892f641261f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:06.321161 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-06827432-9267-47a3-a969-6f7da17bcb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:06.395648 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1318/2621] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:06.400169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d45e0-dbca-4584-bf11-e4c1a32fb5c0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.453382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:06.466352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d45e0-dbca-4584-bf11-e4c1a32fb5c0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.471435 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf63633-e5e9-4d24-a4a7-60ea4d2a64d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.472149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf63633-e5e9-4d24-a4a7-60ea4d2a64d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.472428 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf63633-e5e9-4d24-a4a7-60ea4d2a64d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.472702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf63633-e5e9-4d24-a4a7-60ea4d2a64d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.473422 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf63633-e5e9-4d24-a4a7-60ea4d2a64d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.475308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf63633-e5e9-4d24-a4a7-60ea4d2a64d9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.503903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1304/2622] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:06 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:06.509106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06827432-9267-47a3-a969-6f7da17bcb5d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.523312 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06827432-9267-47a3-a969-6f7da17bcb5d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.527264 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:06.584728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb18f0-34ef-4a51-9577-0538cea56e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.585295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb18f0-34ef-4a51-9577-0538cea56e26 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.585576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb18f0-34ef-4a51-9577-0538cea56e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.585850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb18f0-34ef-4a51-9577-0538cea56e26 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.586537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb18f0-34ef-4a51-9577-0538cea56e26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.588560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb18f0-34ef-4a51-9577-0538cea56e26 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.607750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1319/2623] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:06 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:06.611835 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf63633-e5e9-4d24-a4a7-60ea4d2a64d9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.622303 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf63633-e5e9-4d24-a4a7-60ea4d2a64d9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.627706 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a039ba4-699d-4d8a-9ad5-75b1799b089f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.628366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a039ba4-699d-4d8a-9ad5-75b1799b089f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.628649 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a039ba4-699d-4d8a-9ad5-75b1799b089f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.628923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a039ba4-699d-4d8a-9ad5-75b1799b089f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.629642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a039ba4-699d-4d8a-9ad5-75b1799b089f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.631221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a039ba4-699d-4d8a-9ad5-75b1799b089f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.642192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1305/2624] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:06.646174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb18f0-34ef-4a51-9577-0538cea56e26 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.655989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb18f0-34ef-4a51-9577-0538cea56e26 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.660638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eedcf6-cd12-48d0-98cf-c273b2aebd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.661293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eedcf6-cd12-48d0-98cf-c273b2aebd3d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.661577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eedcf6-cd12-48d0-98cf-c273b2aebd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.661858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eedcf6-cd12-48d0-98cf-c273b2aebd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.662619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eedcf6-cd12-48d0-98cf-c273b2aebd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.664251 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eedcf6-cd12-48d0-98cf-c273b2aebd3d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.677305 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1320/2625] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:06.682587 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a039ba4-699d-4d8a-9ad5-75b1799b089f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.695509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a039ba4-699d-4d8a-9ad5-75b1799b089f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.700293 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ba1b-c9ac-480d-afe8-4608b6beac01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.701245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ba1b-c9ac-480d-afe8-4608b6beac01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.701534 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ba1b-c9ac-480d-afe8-4608b6beac01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.701821 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ba1b-c9ac-480d-afe8-4608b6beac01 admin admin] PATH_INFO: `/v3/projects/0682b7ebadc94ffb9488632e2d2b3bcc/users/723174e4218846d89ae3203fba020f3f/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.717960 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26ba1b-c9ac-480d-afe8-4608b6beac01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0682b7ebadc94ffb9488632e2d2b3bcc, user_id=723174e4218846d89ae3203fba020f3f, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.720759 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26ba1b-c9ac-480d-afe8-4608b6beac01 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.735221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1306/2626] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:06 2022] PUT /identity/v3/projects/0682b7ebadc94ffb9488632e2d2b3bcc/users/723174e4218846d89ae3203fba020f3f/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:06.746281 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2da0b2-aae4-4f7e-92d7-a07d291e2e54 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.746576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2da0b2-aae4-4f7e-92d7-a07d291e2e54 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.746860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2da0b2-aae4-4f7e-92d7-a07d291e2e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.774444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fd2da0b2-aae4-4f7e-92d7-a07d291e2e54 None None] MFA Rules not processed for user `723174e4218846d89ae3203fba020f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:06.775647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd2da0b2-aae4-4f7e-92d7-a07d291e2e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:06.855085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1321/2627] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:06.860169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ba1b-c9ac-480d-afe8-4608b6beac01 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.926025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ba1b-c9ac-480d-afe8-4608b6beac01 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.931094 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997faaa-8e70-4b71-9384-b74a8660713f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.931795 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997faaa-8e70-4b71-9384-b74a8660713f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.932085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997faaa-8e70-4b71-9384-b74a8660713f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.932382 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997faaa-8e70-4b71-9384-b74a8660713f admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.933072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997faaa-8e70-4b71-9384-b74a8660713f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.934984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997faaa-8e70-4b71-9384-b74a8660713f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.961292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1307/2628] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:06 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:06.965350 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2da0b2-aae4-4f7e-92d7-a07d291e2e54 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.977882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2da0b2-aae4-4f7e-92d7-a07d291e2e54 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:07.065953 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e30be6-0c1c-4bf3-90e5-54d80aa9c7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:07.066675 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e30be6-0c1c-4bf3-90e5-54d80aa9c7ea admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:07.066904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e30be6-0c1c-4bf3-90e5-54d80aa9c7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:07.067238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e30be6-0c1c-4bf3-90e5-54d80aa9c7ea admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:07.068012 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e30be6-0c1c-4bf3-90e5-54d80aa9c7ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:07.069873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e30be6-0c1c-4bf3-90e5-54d80aa9c7ea admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:07.095361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1322/2629] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:06 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:07.099214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997faaa-8e70-4b71-9384-b74a8660713f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:07.110656 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997faaa-8e70-4b71-9384-b74a8660713f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:07.115307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e5713-0267-4d47-a8c4-8faa43de6650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:07.116004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e5713-0267-4d47-a8c4-8faa43de6650 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:07.116287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e5713-0267-4d47-a8c4-8faa43de6650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:07.116564 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e5713-0267-4d47-a8c4-8faa43de6650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:07.117372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e5713-0267-4d47-a8c4-8faa43de6650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:07.119500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e5713-0267-4d47-a8c4-8faa43de6650 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:07.130686 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1308/2630] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:07.134862 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e30be6-0c1c-4bf3-90e5-54d80aa9c7ea admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:07.141333 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:07.145502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e30be6-0c1c-4bf3-90e5-54d80aa9c7ea admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:07.151091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86051821-8bf6-481b-a892-3f4429b78995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:07.152005 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86051821-8bf6-481b-a892-3f4429b78995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:07.152288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86051821-8bf6-481b-a892-3f4429b78995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:07.152565 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86051821-8bf6-481b-a892-3f4429b78995 admin admin] PATH_INFO: `/v3/projects/b7653521febd4f43800a8c8318329c94/users/e2b258ef35dc45eca7a1235850fb27d9/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:07.167585 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86051821-8bf6-481b-a892-3f4429b78995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7653521febd4f43800a8c8318329c94, user_id=e2b258ef35dc45eca7a1235850fb27d9, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:07.170265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86051821-8bf6-481b-a892-3f4429b78995 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:07.183088 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1323/2631] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:07 2022] PUT /identity/v3/projects/b7653521febd4f43800a8c8318329c94/users/e2b258ef35dc45eca7a1235850fb27d9/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:07.186915 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e5713-0267-4d47-a8c4-8faa43de6650 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:07.196934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e5713-0267-4d47-a8c4-8faa43de6650 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:07.255553 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b17258-fe97-4cbe-8d43-bf8070c8aeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:07.256254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17258-fe97-4cbe-8d43-bf8070c8aeeb admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:07.256536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17258-fe97-4cbe-8d43-bf8070c8aeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:07.256810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17258-fe97-4cbe-8d43-bf8070c8aeeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:07.257533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b17258-fe97-4cbe-8d43-bf8070c8aeeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:07.259139 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b17258-fe97-4cbe-8d43-bf8070c8aeeb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:07.269224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1309/2632] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:07.273697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86051821-8bf6-481b-a892-3f4429b78995 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:07.284620 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86051821-8bf6-481b-a892-3f4429b78995 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:07.289288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34141e59-aaaf-47c7-aaad-b0a2c83e700a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:07.290148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34141e59-aaaf-47c7-aaad-b0a2c83e700a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:07.290435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34141e59-aaaf-47c7-aaad-b0a2c83e700a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:07.290718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34141e59-aaaf-47c7-aaad-b0a2c83e700a admin admin] PATH_INFO: `/v3/projects/b7653521febd4f43800a8c8318329c94/users/e2b258ef35dc45eca7a1235850fb27d9/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:07.292976 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34141e59-aaaf-47c7-aaad-b0a2c83e700a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7653521febd4f43800a8c8318329c94, user_id=e2b258ef35dc45eca7a1235850fb27d9, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:07.295633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34141e59-aaaf-47c7-aaad-b0a2c83e700a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:07.307001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1324/2633] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:07 2022] PUT /identity/v3/projects/b7653521febd4f43800a8c8318329c94/users/e2b258ef35dc45eca7a1235850fb27d9/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:07.316999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a6d5d-0632-4411-aac4-136ee880fb58 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:07.317292 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a6d5d-0632-4411-aac4-136ee880fb58 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:07.317574 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a6d5d-0632-4411-aac4-136ee880fb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:07.336256 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:07.344232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9a4a6d5d-0632-4411-aac4-136ee880fb58 None None] MFA Rules not processed for user `e2b258ef35dc45eca7a1235850fb27d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:07.345908 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9a4a6d5d-0632-4411-aac4-136ee880fb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:07.418612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1310/2634] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:07.429139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34141e59-aaaf-47c7-aaad-b0a2c83e700a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:07.431006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-34141e59-aaaf-47c7-aaad-b0a2c83e700a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:07.490505 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34141e59-aaaf-47c7-aaad-b0a2c83e700a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:07.494334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d3c2e-3068-424b-bbb9-32ece958442b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:07.494986 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d3c2e-3068-424b-bbb9-32ece958442b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:07.495271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d3c2e-3068-424b-bbb9-32ece958442b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:07.495553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d3c2e-3068-424b-bbb9-32ece958442b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:07.504710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d3c2e-3068-424b-bbb9-32ece958442b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:07.506137 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d3c2e-3068-424b-bbb9-32ece958442b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:07.516097 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1325/2635] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:07.994695 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a6d5d-0632-4411-aac4-136ee880fb58 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:07.996159 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9a4a6d5d-0632-4411-aac4-136ee880fb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:08.008683 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a6d5d-0632-4411-aac4-136ee880fb58 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:08.012803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35121566-688c-4b7c-9f57-bff98b1fa4e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:08.013599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35121566-688c-4b7c-9f57-bff98b1fa4e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:08.013784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35121566-688c-4b7c-9f57-bff98b1fa4e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:08.014098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35121566-688c-4b7c-9f57-bff98b1fa4e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:08.074163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35121566-688c-4b7c-9f57-bff98b1fa4e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:08.075599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35121566-688c-4b7c-9f57-bff98b1fa4e8 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:08.085278 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1311/2636] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:13.704565 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d3c2e-3068-424b-bbb9-32ece958442b service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:13.704565 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af2d3c2e-3068-424b-bbb9-32ece958442b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:13.773435 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d3c2e-3068-424b-bbb9-32ece958442b service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:13.778204 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:13.779111 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:13.779397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:13.779714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] PATH_INFO: `/v3/users/566687399b5544de9fc5e11452a5a13b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:13.780902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=566687399b5544de9fc5e11452a5a13b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:13.782826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:13.827757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '566687399b5544de9fc5e11452a5a13b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.828408 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '566687399b5544de9fc5e11452a5a13b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.829637 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '566687399b5544de9fc5e11452a5a13b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.841537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '566687399b5544de9fc5e11452a5a13b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.846693 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:13.848934 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '566687399b5544de9fc5e11452a5a13b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.862530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1326/2637] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:13 2022] DELETE /identity/v3/users/566687399b5544de9fc5e11452a5a13b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:13.867065 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaca95a-752c-4588-b75a-bd91c51ceeb7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:13.867517 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaca95a-752c-4588-b75a-bd91c51ceeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:13.867673 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaca95a-752c-4588-b75a-bd91c51ceeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:13.885323 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-eaaca95a-752c-4588-b75a-bd91c51ceeb7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:13.886572 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eaaca95a-752c-4588-b75a-bd91c51ceeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:13.948408 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1312/2638] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:13.958634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:13.960092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.020022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b4625-a144-44ed-9238-b0239937a2a1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.023701 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a46669-d697-4bb4-9779-8c7b15420822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.024355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a46669-d697-4bb4-9779-8c7b15420822 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.024636 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a46669-d697-4bb4-9779-8c7b15420822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.024916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a46669-d697-4bb4-9779-8c7b15420822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.036312 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a46669-d697-4bb4-9779-8c7b15420822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.037757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a46669-d697-4bb4-9779-8c7b15420822 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.061426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1327/2639] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:14.192985 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaca95a-752c-4588-b75a-bd91c51ceeb7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.194934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eaaca95a-752c-4588-b75a-bd91c51ceeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.254304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:14.272549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaca95a-752c-4588-b75a-bd91c51ceeb7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.276087 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a5261-6f5c-4897-a86d-be6c9b61c3aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.276741 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a5261-6f5c-4897-a86d-be6c9b61c3aa service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.277021 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a5261-6f5c-4897-a86d-be6c9b61c3aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.277327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a5261-6f5c-4897-a86d-be6c9b61c3aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.279143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d22a5261-6f5c-4897-a86d-be6c9b61c3aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.344631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a5261-6f5c-4897-a86d-be6c9b61c3aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.345992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a5261-6f5c-4897-a86d-be6c9b61c3aa service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.348103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a46669-d697-4bb4-9779-8c7b15420822 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.349553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-09a46669-d697-4bb4-9779-8c7b15420822 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.357698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1313/2640] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:14.369688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a46669-d697-4bb4-9779-8c7b15420822 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.375020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ff4a3-977e-4353-b1b3-984435990f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.376500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ff4a3-977e-4353-b1b3-984435990f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.376812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ff4a3-977e-4353-b1b3-984435990f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.377085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ff4a3-977e-4353-b1b3-984435990f11 admin admin] PATH_INFO: `/v3/projects/2c2f11f9f5cc4b04bb72e54fc5de2e0d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.378364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ff4a3-977e-4353-b1b3-984435990f11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c2f11f9f5cc4b04bb72e54fc5de2e0d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.380375 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ff4a3-977e-4353-b1b3-984435990f11 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.439110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a85ff4a3-977e-4353-b1b3-984435990f11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c2f11f9f5cc4b04bb72e54fc5de2e0d', 'request_id': 'req-a85ff4a3-977e-4353-b1b3-984435990f11'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.445698 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a85ff4a3-977e-4353-b1b3-984435990f11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c2f11f9f5cc4b04bb72e54fc5de2e0d', 'request_id': 'req-a85ff4a3-977e-4353-b1b3-984435990f11'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.451319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a85ff4a3-977e-4353-b1b3-984435990f11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c2f11f9f5cc4b04bb72e54fc5de2e0d', 'request_id': 'req-a85ff4a3-977e-4353-b1b3-984435990f11'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.462738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a85ff4a3-977e-4353-b1b3-984435990f11 admin admin] The token cache is being invalidate because project 2c2f11f9f5cc4b04bb72e54fc5de2e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:14.463035 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a85ff4a3-977e-4353-b1b3-984435990f11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.463728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a85ff4a3-977e-4353-b1b3-984435990f11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.465306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1328/2641] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:14 2022] DELETE /identity/v3/projects/2c2f11f9f5cc4b04bb72e54fc5de2e0d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:14.470779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108417d-c5ad-4140-8152-34c7266bd0cb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.471240 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108417d-c5ad-4140-8152-34c7266bd0cb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.471598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108417d-c5ad-4140-8152-34c7266bd0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.490538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7108417d-c5ad-4140-8152-34c7266bd0cb None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:14.491875 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7108417d-c5ad-4140-8152-34c7266bd0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.554509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1314/2642] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:14.563302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45f371-cb53-4003-8f66-76b3b8f6c796 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.563615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45f371-cb53-4003-8f66-76b3b8f6c796 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.563904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45f371-cb53-4003-8f66-76b3b8f6c796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.590207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fb45f371-cb53-4003-8f66-76b3b8f6c796 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:14.591460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fb45f371-cb53-4003-8f66-76b3b8f6c796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.600270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1329/2643] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:14.604588 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108417d-c5ad-4140-8152-34c7266bd0cb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.614657 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108417d-c5ad-4140-8152-34c7266bd0cb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.619240 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2352b-2dd2-4e67-98a4-59d61c6344a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.619925 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2352b-2dd2-4e67-98a4-59d61c6344a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.620205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2352b-2dd2-4e67-98a4-59d61c6344a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.620477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2352b-2dd2-4e67-98a4-59d61c6344a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.621197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2352b-2dd2-4e67-98a4-59d61c6344a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:14.621501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2352b-2dd2-4e67-98a4-59d61c6344a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.623127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2352b-2dd2-4e67-98a4-59d61c6344a1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.635010 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1315/2644] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:14.639473 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45f371-cb53-4003-8f66-76b3b8f6c796 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.649355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45f371-cb53-4003-8f66-76b3b8f6c796 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.654369 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f705ff6-62a8-4834-add6-f5f7950713e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.655073 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f705ff6-62a8-4834-add6-f5f7950713e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.655358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f705ff6-62a8-4834-add6-f5f7950713e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.655635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f705ff6-62a8-4834-add6-f5f7950713e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.656423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f705ff6-62a8-4834-add6-f5f7950713e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.658891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f705ff6-62a8-4834-add6-f5f7950713e6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.688917 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1330/2645] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:14.690345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2352b-2dd2-4e67-98a4-59d61c6344a1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.700806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2352b-2dd2-4e67-98a4-59d61c6344a1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.766902 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26ac0c-62c6-4243-8d42-f2d1fbb04092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.767638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26ac0c-62c6-4243-8d42-f2d1fbb04092 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.767959 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26ac0c-62c6-4243-8d42-f2d1fbb04092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.768240 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26ac0c-62c6-4243-8d42-f2d1fbb04092 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.769177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a26ac0c-62c6-4243-8d42-f2d1fbb04092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.771585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a26ac0c-62c6-4243-8d42-f2d1fbb04092 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.793761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1316/2646] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:14.798790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f705ff6-62a8-4834-add6-f5f7950713e6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.809205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f705ff6-62a8-4834-add6-f5f7950713e6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.814217 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958b780-7fa7-4415-ae64-ce7e2b7bb984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.815040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958b780-7fa7-4415-ae64-ce7e2b7bb984 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.815456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958b780-7fa7-4415-ae64-ce7e2b7bb984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.816250 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958b780-7fa7-4415-ae64-ce7e2b7bb984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.816870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958b780-7fa7-4415-ae64-ce7e2b7bb984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.818576 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958b780-7fa7-4415-ae64-ce7e2b7bb984 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.830037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1331/2647] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:14.833993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26ac0c-62c6-4243-8d42-f2d1fbb04092 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.845161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26ac0c-62c6-4243-8d42-f2d1fbb04092 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.850387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed7236-4083-463b-a9ac-2ea198756530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.850862 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed7236-4083-463b-a9ac-2ea198756530 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.851202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed7236-4083-463b-a9ac-2ea198756530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.851446 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed7236-4083-463b-a9ac-2ea198756530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.852341 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed7236-4083-463b-a9ac-2ea198756530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.853951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed7236-4083-463b-a9ac-2ea198756530 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.855281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:14.870615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1317/2648] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:14.872351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958b780-7fa7-4415-ae64-ce7e2b7bb984 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.882914 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958b780-7fa7-4415-ae64-ce7e2b7bb984 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.887901 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a31d9-5b00-4a6f-bbfc-91b1fae40c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.888856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a31d9-5b00-4a6f-bbfc-91b1fae40c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.889196 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a31d9-5b00-4a6f-bbfc-91b1fae40c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.889316 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a31d9-5b00-4a6f-bbfc-91b1fae40c88 admin admin] PATH_INFO: `/v3/projects/14658ba96fdf443dba893ad98b1c0469/users/d6238b3ebf184f939cc0b4bb934b328f/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.904968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a31d9-5b00-4a6f-bbfc-91b1fae40c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14658ba96fdf443dba893ad98b1c0469, user_id=d6238b3ebf184f939cc0b4bb934b328f, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.907785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a31d9-5b00-4a6f-bbfc-91b1fae40c88 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.927801 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1332/2649] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:14 2022] PUT /identity/v3/projects/14658ba96fdf443dba893ad98b1c0469/users/d6238b3ebf184f939cc0b4bb934b328f/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:14.940865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22ce53-7687-436c-8d1e-52f1766faf96 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.941157 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22ce53-7687-436c-8d1e-52f1766faf96 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.941430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22ce53-7687-436c-8d1e-52f1766faf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.966800 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1d22ce53-7687-436c-8d1e-52f1766faf96 None None] MFA Rules not processed for user `d6238b3ebf184f939cc0b4bb934b328f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:14.968059 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1d22ce53-7687-436c-8d1e-52f1766faf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:15.035884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1318/2650] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:15.046937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a31d9-5b00-4a6f-bbfc-91b1fae40c88 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:15.049300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fe5a31d9-5b00-4a6f-bbfc-91b1fae40c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:15.064174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:15.131673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a31d9-5b00-4a6f-bbfc-91b1fae40c88 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:15.137508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b8e2d-dbce-4379-aee7-171efb422ea2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:15.138206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b8e2d-dbce-4379-aee7-171efb422ea2 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.138426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b8e2d-dbce-4379-aee7-171efb422ea2 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.138720 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b8e2d-dbce-4379-aee7-171efb422ea2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.148359 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b8e2d-dbce-4379-aee7-171efb422ea2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:15.149539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b8e2d-dbce-4379-aee7-171efb422ea2 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:15.158361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1333/2651] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:16.480830 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22ce53-7687-436c-8d1e-52f1766faf96 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:16.482282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1d22ce53-7687-436c-8d1e-52f1766faf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:16.549306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22ce53-7687-436c-8d1e-52f1766faf96 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:16.554133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:16.555005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:16.555284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:16.555554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] PATH_INFO: `/v3/users/5069b527255c458ead325892f641261f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:16.556774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5069b527255c458ead325892f641261f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:16.558776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:16.608199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5069b527255c458ead325892f641261f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.608798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5069b527255c458ead325892f641261f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.616639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5069b527255c458ead325892f641261f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.629510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5069b527255c458ead325892f641261f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.637441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5069b527255c458ead325892f641261f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.643561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1319/2652] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:16 2022] DELETE /identity/v3/users/5069b527255c458ead325892f641261f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:16.648362 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b460f-3ad5-4de7-9871-4fa18836d5a9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:16.648828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b460f-3ad5-4de7-9871-4fa18836d5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:16.649051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b460f-3ad5-4de7-9871-4fa18836d5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:16.669122 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4d2b460f-3ad5-4de7-9871-4fa18836d5a9 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:16.670436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4d2b460f-3ad5-4de7-9871-4fa18836d5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:16.736202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1334/2653] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:16.743603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:16.745048 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:16.752631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:16.807081 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d378e12-7eed-426f-aaf5-10c76261bc4e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:16.810845 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e7f86-add8-4066-9d79-0752211cf669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:16.811526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e7f86-add8-4066-9d79-0752211cf669 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:16.811841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e7f86-add8-4066-9d79-0752211cf669 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:16.812121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e7f86-add8-4066-9d79-0752211cf669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:16.821169 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e7f86-add8-4066-9d79-0752211cf669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:16.822595 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e7f86-add8-4066-9d79-0752211cf669 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:16.832043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1320/2654] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:17.101085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b460f-3ad5-4de7-9871-4fa18836d5a9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.102552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4d2b460f-3ad5-4de7-9871-4fa18836d5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.116510 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b460f-3ad5-4de7-9871-4fa18836d5a9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.121113 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.121998 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.122278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.122553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] PATH_INFO: `/v3/projects/64572285cac84b07b702cebf00d1722f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.123894 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64572285cac84b07b702cebf00d1722f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.125743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.176190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64572285cac84b07b702cebf00d1722f', 'request_id': 'req-fd701956-997a-4007-9272-b42341f5a3c1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.182900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64572285cac84b07b702cebf00d1722f', 'request_id': 'req-fd701956-997a-4007-9272-b42341f5a3c1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.188748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64572285cac84b07b702cebf00d1722f', 'request_id': 'req-fd701956-997a-4007-9272-b42341f5a3c1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.201080 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] The token cache is being invalidate because project 64572285cac84b07b702cebf00d1722f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:17.201373 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.202344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.203550 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1335/2655] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:17 2022] DELETE /identity/v3/projects/64572285cac84b07b702cebf00d1722f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:17.208238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e7f86-add8-4066-9d79-0752211cf669 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.218510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e7f86-add8-4066-9d79-0752211cf669 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.276255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.277160 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.277441 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.277719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] PATH_INFO: `/v3/users/723174e4218846d89ae3203fba020f3f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.279201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=723174e4218846d89ae3203fba020f3f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.281177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.327281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '723174e4218846d89ae3203fba020f3f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.327965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '723174e4218846d89ae3203fba020f3f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.336690 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '723174e4218846d89ae3203fba020f3f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.350948 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '723174e4218846d89ae3203fba020f3f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.357853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '723174e4218846d89ae3203fba020f3f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.363822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1321/2656] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:17 2022] DELETE /identity/v3/users/723174e4218846d89ae3203fba020f3f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:17.632176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.633674 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.646942 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd701956-997a-4007-9272-b42341f5a3c1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.663941 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:17.705022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.706184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.706184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.706964 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] PATH_INFO: `/v3/projects/0682b7ebadc94ffb9488632e2d2b3bcc` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.708228 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0682b7ebadc94ffb9488632e2d2b3bcc)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.710158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.754614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0682b7ebadc94ffb9488632e2d2b3bcc', 'request_id': 'req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.760507 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0682b7ebadc94ffb9488632e2d2b3bcc', 'request_id': 'req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.765882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0682b7ebadc94ffb9488632e2d2b3bcc', 'request_id': 'req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.777692 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] The token cache is being invalidate because project 0682b7ebadc94ffb9488632e2d2b3bcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:17.777978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.778593 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.780142 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1336/2657] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:17 2022] DELETE /identity/v3/projects/0682b7ebadc94ffb9488632e2d2b3bcc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:17.784735 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.786231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.799803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b0535-bf98-4e03-adde-a235e1694487 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.824238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:17.845615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.847059 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.862189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6895316-d9b6-48eb-9c90-711a00b9aeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.863174 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6895316-d9b6-48eb-9c90-711a00b9aeea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.863466 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6895316-d9b6-48eb-9c90-711a00b9aeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.863781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6895316-d9b6-48eb-9c90-711a00b9aeea admin admin] PATH_INFO: `/v3/users/e2b258ef35dc45eca7a1235850fb27d9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.864983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6895316-d9b6-48eb-9c90-711a00b9aeea admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2b258ef35dc45eca7a1235850fb27d9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.867057 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6895316-d9b6-48eb-9c90-711a00b9aeea admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.913295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e8d9e-d90a-4235-92e3-aeff3069b02b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.915467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c6895316-d9b6-48eb-9c90-711a00b9aeea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2b258ef35dc45eca7a1235850fb27d9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.916249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c6895316-d9b6-48eb-9c90-711a00b9aeea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2b258ef35dc45eca7a1235850fb27d9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.918165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.919049 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.919336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.919616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] PATH_INFO: `/v3/users/d6238b3ebf184f939cc0b4bb934b328f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.921016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6238b3ebf184f939cc0b4bb934b328f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.923155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.923764 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c6895316-d9b6-48eb-9c90-711a00b9aeea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2b258ef35dc45eca7a1235850fb27d9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.935903 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c6895316-d9b6-48eb-9c90-711a00b9aeea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2b258ef35dc45eca7a1235850fb27d9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.942930 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c6895316-d9b6-48eb-9c90-711a00b9aeea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2b258ef35dc45eca7a1235850fb27d9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.949775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1322/2658] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:17 2022] DELETE /identity/v3/users/e2b258ef35dc45eca7a1235850fb27d9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:17.973249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6238b3ebf184f939cc0b4bb934b328f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.974031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6238b3ebf184f939cc0b4bb934b328f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.974649 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6238b3ebf184f939cc0b4bb934b328f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.979103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:17.989344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6238b3ebf184f939cc0b4bb934b328f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.996817 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6238b3ebf184f939cc0b4bb934b328f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.010412 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1337/2659] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:17 2022] DELETE /identity/v3/users/d6238b3ebf184f939cc0b4bb934b328f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:18.015133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d3aa3-cded-4a1b-bff6-3743f820ec33 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.015586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d3aa3-cded-4a1b-bff6-3743f820ec33 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.015975 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d3aa3-cded-4a1b-bff6-3743f820ec33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.036550 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bc3d3aa3-cded-4a1b-bff6-3743f820ec33 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:18.037900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bc3d3aa3-cded-4a1b-bff6-3743f820ec33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.103767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1323/2660] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:18.335124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.336865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.396644 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c76ae-1f9e-48fc-a551-02618a3c78fe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.400339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47f1a5-0d09-4654-aff8-aa63c79a876e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.400985 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47f1a5-0d09-4654-aff8-aa63c79a876e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.401270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47f1a5-0d09-4654-aff8-aa63c79a876e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.401549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47f1a5-0d09-4654-aff8-aa63c79a876e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.410629 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47f1a5-0d09-4654-aff8-aa63c79a876e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.412098 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47f1a5-0d09-4654-aff8-aa63c79a876e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.421993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1338/2661] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:18.474520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d3aa3-cded-4a1b-bff6-3743f820ec33 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.475983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bc3d3aa3-cded-4a1b-bff6-3743f820ec33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.487641 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d3aa3-cded-4a1b-bff6-3743f820ec33 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.492720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fd88b-5dab-40a2-a46f-c4e8c4552eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.493600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fd88b-5dab-40a2-a46f-c4e8c4552eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.493881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fd88b-5dab-40a2-a46f-c4e8c4552eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.494260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fd88b-5dab-40a2-a46f-c4e8c4552eef admin admin] PATH_INFO: `/v3/projects/b7653521febd4f43800a8c8318329c94` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.495469 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229fd88b-5dab-40a2-a46f-c4e8c4552eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7653521febd4f43800a8c8318329c94)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.497398 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229fd88b-5dab-40a2-a46f-c4e8c4552eef admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.541701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:18.549841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-229fd88b-5dab-40a2-a46f-c4e8c4552eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7653521febd4f43800a8c8318329c94', 'request_id': 'req-229fd88b-5dab-40a2-a46f-c4e8c4552eef'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.557345 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-229fd88b-5dab-40a2-a46f-c4e8c4552eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7653521febd4f43800a8c8318329c94', 'request_id': 'req-229fd88b-5dab-40a2-a46f-c4e8c4552eef'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.563831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-229fd88b-5dab-40a2-a46f-c4e8c4552eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7653521febd4f43800a8c8318329c94', 'request_id': 'req-229fd88b-5dab-40a2-a46f-c4e8c4552eef'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.575598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-229fd88b-5dab-40a2-a46f-c4e8c4552eef admin admin] The token cache is being invalidate because project b7653521febd4f43800a8c8318329c94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:18.575899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-229fd88b-5dab-40a2-a46f-c4e8c4552eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.576598 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-229fd88b-5dab-40a2-a46f-c4e8c4552eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.578488 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1324/2662] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:18 2022] DELETE /identity/v3/projects/b7653521febd4f43800a8c8318329c94 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:18.585218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebdecc-9bdb-4013-b172-4a43c9ef08d0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.585635 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebdecc-9bdb-4013-b172-4a43c9ef08d0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.586126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebdecc-9bdb-4013-b172-4a43c9ef08d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.606615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-23ebdecc-9bdb-4013-b172-4a43c9ef08d0 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:18.607856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-23ebdecc-9bdb-4013-b172-4a43c9ef08d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.670117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1339/2663] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:18.679052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569709c-297b-4406-94d2-d7347828b00b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.679418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569709c-297b-4406-94d2-d7347828b00b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.679817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569709c-297b-4406-94d2-d7347828b00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.699807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c569709c-297b-4406-94d2-d7347828b00b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:18.701184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c569709c-297b-4406-94d2-d7347828b00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.710561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1325/2664] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:18.715269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebdecc-9bdb-4013-b172-4a43c9ef08d0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.715437 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569709c-297b-4406-94d2-d7347828b00b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.716695 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c569709c-297b-4406-94d2-d7347828b00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.729315 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebdecc-9bdb-4013-b172-4a43c9ef08d0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.734313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef8830-c208-47ba-b712-9a912a3568fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.735748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef8830-c208-47ba-b712-9a912a3568fc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.736033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef8830-c208-47ba-b712-9a912a3568fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.736318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef8830-c208-47ba-b712-9a912a3568fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.737032 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef8830-c208-47ba-b712-9a912a3568fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:18.737161 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569709c-297b-4406-94d2-d7347828b00b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.737344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef8830-c208-47ba-b712-9a912a3568fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.739332 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef8830-c208-47ba-b712-9a912a3568fc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.743089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c936b17-a5eb-46fd-928c-189e95ddcf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.744236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c936b17-a5eb-46fd-928c-189e95ddcf93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.744713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c936b17-a5eb-46fd-928c-189e95ddcf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.744983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c936b17-a5eb-46fd-928c-189e95ddcf93 admin admin] PATH_INFO: `/v3/projects/14658ba96fdf443dba893ad98b1c0469` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.746868 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c936b17-a5eb-46fd-928c-189e95ddcf93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14658ba96fdf443dba893ad98b1c0469)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.748822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c936b17-a5eb-46fd-928c-189e95ddcf93 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.755470 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1340/2665] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:18.756432 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef8830-c208-47ba-b712-9a912a3568fc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.766218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef8830-c208-47ba-b712-9a912a3568fc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.771064 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89420f02-64fc-46cc-bb4c-6dd5c5662618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.771727 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89420f02-64fc-46cc-bb4c-6dd5c5662618 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.772015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89420f02-64fc-46cc-bb4c-6dd5c5662618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.772290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89420f02-64fc-46cc-bb4c-6dd5c5662618 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.772974 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89420f02-64fc-46cc-bb4c-6dd5c5662618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.774858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89420f02-64fc-46cc-bb4c-6dd5c5662618 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.797197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8c936b17-a5eb-46fd-928c-189e95ddcf93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14658ba96fdf443dba893ad98b1c0469', 'request_id': 'req-8c936b17-a5eb-46fd-928c-189e95ddcf93'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.801111 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1341/2666] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:18.804605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8c936b17-a5eb-46fd-928c-189e95ddcf93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14658ba96fdf443dba893ad98b1c0469', 'request_id': 'req-8c936b17-a5eb-46fd-928c-189e95ddcf93'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.805650 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89420f02-64fc-46cc-bb4c-6dd5c5662618 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.810491 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8c936b17-a5eb-46fd-928c-189e95ddcf93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14658ba96fdf443dba893ad98b1c0469', 'request_id': 'req-8c936b17-a5eb-46fd-928c-189e95ddcf93'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.822239 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8c936b17-a5eb-46fd-928c-189e95ddcf93 admin admin] The token cache is being invalidate because project 14658ba96fdf443dba893ad98b1c0469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:18.822533 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8c936b17-a5eb-46fd-928c-189e95ddcf93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.823191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8c936b17-a5eb-46fd-928c-189e95ddcf93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.824794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1326/2667] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:18 2022] DELETE /identity/v3/projects/14658ba96fdf443dba893ad98b1c0469 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:18.829520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7e289-7dbd-454f-a767-a65d06c22dc7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.829810 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7e289-7dbd-454f-a767-a65d06c22dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.830099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7e289-7dbd-454f-a767-a65d06c22dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.847864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-84f7e289-7dbd-454f-a767-a65d06c22dc7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:18.849066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-84f7e289-7dbd-454f-a767-a65d06c22dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.851613 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:18.874594 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89420f02-64fc-46cc-bb4c-6dd5c5662618 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.879834 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0cdf5-efbd-4a09-bb4f-9756c40ce267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.880498 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0cdf5-efbd-4a09-bb4f-9756c40ce267 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.880780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0cdf5-efbd-4a09-bb4f-9756c40ce267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.881069 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0cdf5-efbd-4a09-bb4f-9756c40ce267 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.881754 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0cdf5-efbd-4a09-bb4f-9756c40ce267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.883688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0cdf5-efbd-4a09-bb4f-9756c40ce267 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.905205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1342/2668] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:18.908955 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0cdf5-efbd-4a09-bb4f-9756c40ce267 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.910524 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b6a0cdf5-efbd-4a09-bb4f-9756c40ce267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.911638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1327/2669] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:18.919841 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c2777-5755-40b6-9e91-266fed67d969 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.920133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c2777-5755-40b6-9e91-266fed67d969 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.920412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c2777-5755-40b6-9e91-266fed67d969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.923337 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0cdf5-efbd-4a09-bb4f-9756c40ce267 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.928762 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434c7fd-53e7-4855-84dd-f86006d18f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.928762 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434c7fd-53e7-4855-84dd-f86006d18f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.929030 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434c7fd-53e7-4855-84dd-f86006d18f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.929254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434c7fd-53e7-4855-84dd-f86006d18f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.929986 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434c7fd-53e7-4855-84dd-f86006d18f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.931577 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434c7fd-53e7-4855-84dd-f86006d18f27 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.939353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-303c2777-5755-40b6-9e91-266fed67d969 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:18.940592 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-303c2777-5755-40b6-9e91-266fed67d969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.942173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1343/2670] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:18.946244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434c7fd-53e7-4855-84dd-f86006d18f27 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.956105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434c7fd-53e7-4855-84dd-f86006d18f27 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.956735 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1328/2671] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:18.960659 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c2777-5755-40b6-9e91-266fed67d969 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.964018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541f5be-62e1-4b52-b1cd-0ed7590b84e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.964680 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541f5be-62e1-4b52-b1cd-0ed7590b84e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.964971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541f5be-62e1-4b52-b1cd-0ed7590b84e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.965256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541f5be-62e1-4b52-b1cd-0ed7590b84e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.966226 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541f5be-62e1-4b52-b1cd-0ed7590b84e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.967874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541f5be-62e1-4b52-b1cd-0ed7590b84e4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.970216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c2777-5755-40b6-9e91-266fed67d969 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.974697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f1ffd-2c72-497d-8bc3-0540d1d62931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.975328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f1ffd-2c72-497d-8bc3-0540d1d62931 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.975605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f1ffd-2c72-497d-8bc3-0540d1d62931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.975909 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f1ffd-2c72-497d-8bc3-0540d1d62931 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.976628 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f1ffd-2c72-497d-8bc3-0540d1d62931 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:18.976932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f1ffd-2c72-497d-8bc3-0540d1d62931 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.978503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f1ffd-2c72-497d-8bc3-0540d1d62931 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.979261 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1344/2672] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:18.983549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541f5be-62e1-4b52-b1cd-0ed7590b84e4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.994694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1329/2673] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:11:18.994876 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541f5be-62e1-4b52-b1cd-0ed7590b84e4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.995712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f1ffd-2c72-497d-8bc3-0540d1d62931 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.998461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b28d32-058f-45f8-a7f8-c4bd5e7a4970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.999336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b28d32-058f-45f8-a7f8-c4bd5e7a4970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.999622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b28d32-058f-45f8-a7f8-c4bd5e7a4970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.999929 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b28d32-058f-45f8-a7f8-c4bd5e7a4970 admin admin] PATH_INFO: `/v3/projects/4cedc8008734430698d420fc194d706e/users/062a8da471df4a74b989d2fe3e908244/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.005977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f1ffd-2c72-497d-8bc3-0540d1d62931 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.010642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca13a6-bc8d-4e8c-83dd-f69cc234f70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.011284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca13a6-bc8d-4e8c-83dd-f69cc234f70a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.011574 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca13a6-bc8d-4e8c-83dd-f69cc234f70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.011881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca13a6-bc8d-4e8c-83dd-f69cc234f70a admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.012569 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca13a6-bc8d-4e8c-83dd-f69cc234f70a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.014479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca13a6-bc8d-4e8c-83dd-f69cc234f70a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.014628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b28d32-058f-45f8-a7f8-c4bd5e7a4970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cedc8008734430698d420fc194d706e, user_id=062a8da471df4a74b989d2fe3e908244, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.017316 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b28d32-058f-45f8-a7f8-c4bd5e7a4970 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.030787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1345/2674] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:18 2022] PUT /identity/v3/projects/4cedc8008734430698d420fc194d706e/users/062a8da471df4a74b989d2fe3e908244/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:19.041287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1330/2675] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:19.045144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca13a6-bc8d-4e8c-83dd-f69cc234f70a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.045705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f208a2f-0796-4f8b-a8a2-be18d197b732 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.045996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f208a2f-0796-4f8b-a8a2-be18d197b732 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.046270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f208a2f-0796-4f8b-a8a2-be18d197b732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.051311 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:19.073152 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5f208a2f-0796-4f8b-a8a2-be18d197b732 None None] MFA Rules not processed for user `062a8da471df4a74b989d2fe3e908244`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:19.074389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5f208a2f-0796-4f8b-a8a2-be18d197b732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:19.112077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca13a6-bc8d-4e8c-83dd-f69cc234f70a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.116821 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3ac94-a494-4142-9323-f6f8a4d4cf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.117490 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3ac94-a494-4142-9323-f6f8a4d4cf40 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.117773 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3ac94-a494-4142-9323-f6f8a4d4cf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.118052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3ac94-a494-4142-9323-f6f8a4d4cf40 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.118747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3ac94-a494-4142-9323-f6f8a4d4cf40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.120713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3ac94-a494-4142-9323-f6f8a4d4cf40 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.140095 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1346/2677] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:19.140373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1331/2677] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:19 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:19.144481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3ac94-a494-4142-9323-f6f8a4d4cf40 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.144867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f208a2f-0796-4f8b-a8a2-be18d197b732 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.154270 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3ac94-a494-4142-9323-f6f8a4d4cf40 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.154860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f208a2f-0796-4f8b-a8a2-be18d197b732 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.159584 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad8c78-ebb0-417d-9174-f355030eeca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.160005 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4c4df-157c-4443-b94b-22da0ce0e75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.160254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad8c78-ebb0-417d-9174-f355030eeca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.160536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad8c78-ebb0-417d-9174-f355030eeca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.160653 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4c4df-157c-4443-b94b-22da0ce0e75a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.160812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad8c78-ebb0-417d-9174-f355030eeca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.160919 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4c4df-157c-4443-b94b-22da0ce0e75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.161180 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4c4df-157c-4443-b94b-22da0ce0e75a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.161494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad8c78-ebb0-417d-9174-f355030eeca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.161902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4c4df-157c-4443-b94b-22da0ce0e75a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.163406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad8c78-ebb0-417d-9174-f355030eeca8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.163536 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4c4df-157c-4443-b94b-22da0ce0e75a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.173872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1347/2678] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:19.177665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4c4df-157c-4443-b94b-22da0ce0e75a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.187699 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4c4df-157c-4443-b94b-22da0ce0e75a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.192191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1332/2679] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:19 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:19.192942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad8c78-ebb0-417d-9174-f355030eeca8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.245611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb25ebc-7b77-4e31-bc1f-0f9c6a2b2298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.246271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb25ebc-7b77-4e31-bc1f-0f9c6a2b2298 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.246549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb25ebc-7b77-4e31-bc1f-0f9c6a2b2298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.246823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb25ebc-7b77-4e31-bc1f-0f9c6a2b2298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.247541 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb25ebc-7b77-4e31-bc1f-0f9c6a2b2298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.249146 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb25ebc-7b77-4e31-bc1f-0f9c6a2b2298 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.250512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:19.255986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad8c78-ebb0-417d-9174-f355030eeca8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.260869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70f3fe-7a16-40a3-bdb7-ad4e1fad9b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.261040 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1348/2680] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:19.261692 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70f3fe-7a16-40a3-bdb7-ad4e1fad9b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.262015 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70f3fe-7a16-40a3-bdb7-ad4e1fad9b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.262291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70f3fe-7a16-40a3-bdb7-ad4e1fad9b6f admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.262965 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70f3fe-7a16-40a3-bdb7-ad4e1fad9b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.264952 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb25ebc-7b77-4e31-bc1f-0f9c6a2b2298 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.265304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70f3fe-7a16-40a3-bdb7-ad4e1fad9b6f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.274965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb25ebc-7b77-4e31-bc1f-0f9c6a2b2298 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.280429 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40bbfa-940e-418a-81c9-abe015ca7330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.281308 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40bbfa-940e-418a-81c9-abe015ca7330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.281598 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40bbfa-940e-418a-81c9-abe015ca7330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.282433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40bbfa-940e-418a-81c9-abe015ca7330 admin admin] PATH_INFO: `/v3/projects/6bd70e3417224f9c8078be5b95c558a2/users/cd1b6f431ef34605a1053cf87191186d/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.286285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1333/2681] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:19 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:19.290103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70f3fe-7a16-40a3-bdb7-ad4e1fad9b6f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.299704 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40bbfa-940e-418a-81c9-abe015ca7330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd70e3417224f9c8078be5b95c558a2, user_id=cd1b6f431ef34605a1053cf87191186d, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.300031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70f3fe-7a16-40a3-bdb7-ad4e1fad9b6f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.302425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40bbfa-940e-418a-81c9-abe015ca7330 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.305324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57ad9b-b0fd-4002-bd6b-2e38c8449640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.305984 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57ad9b-b0fd-4002-bd6b-2e38c8449640 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.306267 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57ad9b-b0fd-4002-bd6b-2e38c8449640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.306544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57ad9b-b0fd-4002-bd6b-2e38c8449640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.307274 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57ad9b-b0fd-4002-bd6b-2e38c8449640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.308878 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57ad9b-b0fd-4002-bd6b-2e38c8449640 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.318864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1349/2682] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:19 2022] PUT /identity/v3/projects/6bd70e3417224f9c8078be5b95c558a2/users/cd1b6f431ef34605a1053cf87191186d/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:19.320547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1334/2683] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:19.324424 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40bbfa-940e-418a-81c9-abe015ca7330 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.330052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763c23-1ef2-467b-ad55-c7eb80c609f8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.330342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763c23-1ef2-467b-ad55-c7eb80c609f8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.330619 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763c23-1ef2-467b-ad55-c7eb80c609f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.367185 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7f763c23-1ef2-467b-ad55-c7eb80c609f8 None None] MFA Rules not processed for user `cd1b6f431ef34605a1053cf87191186d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:19.368430 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7f763c23-1ef2-467b-ad55-c7eb80c609f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:19.389730 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40bbfa-940e-418a-81c9-abe015ca7330 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.395238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cfb23-3cbe-46d7-a61f-47fc882c70b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.396173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cfb23-3cbe-46d7-a61f-47fc882c70b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.396456 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cfb23-3cbe-46d7-a61f-47fc882c70b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.396733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cfb23-3cbe-46d7-a61f-47fc882c70b8 admin admin] PATH_INFO: `/v3/projects/29430b58143c42319a14201ff930099d/users/305f9048cd6548959c3438acd26dac88/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.409132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:19.412013 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cfb23-3cbe-46d7-a61f-47fc882c70b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29430b58143c42319a14201ff930099d, user_id=305f9048cd6548959c3438acd26dac88, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.415018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cfb23-3cbe-46d7-a61f-47fc882c70b8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.427984 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1350/2684] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:19 2022] PUT /identity/v3/projects/29430b58143c42319a14201ff930099d/users/305f9048cd6548959c3438acd26dac88/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:19.431827 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cfb23-3cbe-46d7-a61f-47fc882c70b8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.435718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1335/2685] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:19.446307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f763c23-1ef2-467b-ad55-c7eb80c609f8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.447702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7f763c23-1ef2-467b-ad55-c7eb80c609f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:19.497574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cfb23-3cbe-46d7-a61f-47fc882c70b8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.502384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889116b-f22d-4d7e-9b04-e8d526a615dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.503057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889116b-f22d-4d7e-9b04-e8d526a615dc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.503342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889116b-f22d-4d7e-9b04-e8d526a615dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.503622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889116b-f22d-4d7e-9b04-e8d526a615dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.504389 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889116b-f22d-4d7e-9b04-e8d526a615dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.504924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f763c23-1ef2-467b-ad55-c7eb80c609f8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.506241 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889116b-f22d-4d7e-9b04-e8d526a615dc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.516905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1351/2686] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:19.521167 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889116b-f22d-4d7e-9b04-e8d526a615dc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.530780 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889116b-f22d-4d7e-9b04-e8d526a615dc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.536033 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e40a7-a81e-4890-97f8-7bbe951cf66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.536912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e40a7-a81e-4890-97f8-7bbe951cf66e admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.537203 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e40a7-a81e-4890-97f8-7bbe951cf66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.537482 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e40a7-a81e-4890-97f8-7bbe951cf66e admin admin] PATH_INFO: `/v3/projects/29430b58143c42319a14201ff930099d/users/305f9048cd6548959c3438acd26dac88/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.539473 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e40a7-a81e-4890-97f8-7bbe951cf66e admin admin] RBAC: Authorizing `identity:create_grant(project_id=29430b58143c42319a14201ff930099d, user_id=305f9048cd6548959c3438acd26dac88, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.542126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e40a7-a81e-4890-97f8-7bbe951cf66e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.552561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1352/2687] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:19 2022] PUT /identity/v3/projects/29430b58143c42319a14201ff930099d/users/305f9048cd6548959c3438acd26dac88/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:19.562228 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf49dc-c0f0-4fd8-8358-d73767e70960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.562575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc85a7-e4d8-4755-8d1f-26f9aae7957c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.562867 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc85a7-e4d8-4755-8d1f-26f9aae7957c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.562980 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf49dc-c0f0-4fd8-8358-d73767e70960 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.563172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc85a7-e4d8-4755-8d1f-26f9aae7957c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.563287 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf49dc-c0f0-4fd8-8358-d73767e70960 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.563461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf49dc-c0f0-4fd8-8358-d73767e70960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.590872 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5cfc85a7-e4d8-4755-8d1f-26f9aae7957c None None] MFA Rules not processed for user `305f9048cd6548959c3438acd26dac88`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:19.592122 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5cfc85a7-e4d8-4755-8d1f-26f9aae7957c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:19.619011 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:19.620912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf49dc-c0f0-4fd8-8358-d73767e70960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.622858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf49dc-c0f0-4fd8-8358-d73767e70960 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.632102 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1336/2688] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:19.665018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1353/2689] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:19.675718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf49dc-c0f0-4fd8-8358-d73767e70960 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.677113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-01cf49dc-c0f0-4fd8-8358-d73767e70960 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:19.688313 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf49dc-c0f0-4fd8-8358-d73767e70960 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.737159 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d99ae-a9ed-4485-ab95-0c0ed38e8a16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.737807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d99ae-a9ed-4485-ab95-0c0ed38e8a16 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.738086 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d99ae-a9ed-4485-ab95-0c0ed38e8a16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.738359 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d99ae-a9ed-4485-ab95-0c0ed38e8a16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.744654 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:19.749387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d99ae-a9ed-4485-ab95-0c0ed38e8a16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.751348 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d99ae-a9ed-4485-ab95-0c0ed38e8a16 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.761276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1337/2690] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:20.354606 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc85a7-e4d8-4755-8d1f-26f9aae7957c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:20.418394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc85a7-e4d8-4755-8d1f-26f9aae7957c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:20.423519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:20.424417 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:20.424700 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:20.424980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] PATH_INFO: `/v3/users/062a8da471df4a74b989d2fe3e908244` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:20.426165 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=062a8da471df4a74b989d2fe3e908244)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:20.428116 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:20.473297 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '062a8da471df4a74b989d2fe3e908244'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:20.473945 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '062a8da471df4a74b989d2fe3e908244'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:20.474555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '062a8da471df4a74b989d2fe3e908244'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:20.486959 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '062a8da471df4a74b989d2fe3e908244'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:20.493816 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '062a8da471df4a74b989d2fe3e908244'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:20.506611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1354/2691] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:20 2022] DELETE /identity/v3/users/062a8da471df4a74b989d2fe3e908244 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:20.511261 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902a834-f3b0-4499-9bd1-fbf65ce76bed None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:20.511779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902a834-f3b0-4499-9bd1-fbf65ce76bed None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:20.511881 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902a834-f3b0-4499-9bd1-fbf65ce76bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:20.529526 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b902a834-f3b0-4499-9bd1-fbf65ce76bed None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:20.530780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b902a834-f3b0-4499-9bd1-fbf65ce76bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:20.600062 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1338/2692] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:20.609660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:20.611765 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:20.657465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:20.670362 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95999a-703d-4230-ab0e-d68a0c926be0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:20.674711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db2ff3-ab95-4f63-a03c-f149b41c8664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:20.675351 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db2ff3-ab95-4f63-a03c-f149b41c8664 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:20.675633 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db2ff3-ab95-4f63-a03c-f149b41c8664 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:20.675939 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db2ff3-ab95-4f63-a03c-f149b41c8664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:20.684871 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db2ff3-ab95-4f63-a03c-f149b41c8664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:20.686293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db2ff3-ab95-4f63-a03c-f149b41c8664 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:20.695638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1355/2693] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:20.966374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902a834-f3b0-4499-9bd1-fbf65ce76bed None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:20.968131 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b902a834-f3b0-4499-9bd1-fbf65ce76bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:20.981317 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902a834-f3b0-4499-9bd1-fbf65ce76bed None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:20.987346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:20.988341 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:20.988511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:20.988791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] PATH_INFO: `/v3/projects/4cedc8008734430698d420fc194d706e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:20.990080 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cedc8008734430698d420fc194d706e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:20.992003 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:21.041712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cedc8008734430698d420fc194d706e', 'request_id': 'req-f5139821-51af-4b8f-8430-f460506cb3f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.049632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cedc8008734430698d420fc194d706e', 'request_id': 'req-f5139821-51af-4b8f-8430-f460506cb3f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.055593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cedc8008734430698d420fc194d706e', 'request_id': 'req-f5139821-51af-4b8f-8430-f460506cb3f1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.069974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] The token cache is being invalidate because project 4cedc8008734430698d420fc194d706e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:21.070288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.070928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.072504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1339/2694] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:20 2022] DELETE /identity/v3/projects/4cedc8008734430698d420fc194d706e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:21.076372 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db2ff3-ab95-4f63-a03c-f149b41c8664 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:21.086493 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db2ff3-ab95-4f63-a03c-f149b41c8664 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:21.143537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7767251-79ef-48d7-bed0-8955bf89a94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:21.144447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7767251-79ef-48d7-bed0-8955bf89a94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:21.144987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7767251-79ef-48d7-bed0-8955bf89a94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:21.145266 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7767251-79ef-48d7-bed0-8955bf89a94e admin admin] PATH_INFO: `/v3/users/305f9048cd6548959c3438acd26dac88` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:21.146614 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7767251-79ef-48d7-bed0-8955bf89a94e admin admin] RBAC: Authorizing `identity:delete_user(user_id=305f9048cd6548959c3438acd26dac88)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:21.148579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7767251-79ef-48d7-bed0-8955bf89a94e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:21.198862 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7767251-79ef-48d7-bed0-8955bf89a94e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305f9048cd6548959c3438acd26dac88'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.199569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7767251-79ef-48d7-bed0-8955bf89a94e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305f9048cd6548959c3438acd26dac88'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.200224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7767251-79ef-48d7-bed0-8955bf89a94e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305f9048cd6548959c3438acd26dac88'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.213673 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7767251-79ef-48d7-bed0-8955bf89a94e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305f9048cd6548959c3438acd26dac88'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.221399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7767251-79ef-48d7-bed0-8955bf89a94e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305f9048cd6548959c3438acd26dac88'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.234448 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1356/2695] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:21 2022] DELETE /identity/v3/users/305f9048cd6548959c3438acd26dac88 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:21.522888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:21.524351 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:21.537986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5139821-51af-4b8f-8430-f460506cb3f1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:21.594107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:21.595472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:21.595797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:21.596077 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5 admin admin] PATH_INFO: `/v3/projects/29430b58143c42319a14201ff930099d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:21.597261 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29430b58143c42319a14201ff930099d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:21.599180 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:21.620784 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:21.647434 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29430b58143c42319a14201ff930099d', 'request_id': 'req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.654367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29430b58143c42319a14201ff930099d', 'request_id': 'req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.660465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29430b58143c42319a14201ff930099d', 'request_id': 'req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.671249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5 admin admin] The token cache is being invalidate because project 29430b58143c42319a14201ff930099d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:21.671536 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.672209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e787903b-d9f9-4b61-ad54-c0c44a90d8b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:21.673753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1340/2696] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:21 2022] DELETE /identity/v3/projects/29430b58143c42319a14201ff930099d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:21.678696 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02da96d-3eca-492f-9d49-41db0dad0fc7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:21.679153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02da96d-3eca-492f-9d49-41db0dad0fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:21.679248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02da96d-3eca-492f-9d49-41db0dad0fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:21.696954 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f02da96d-3eca-492f-9d49-41db0dad0fc7 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:21.698154 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f02da96d-3eca-492f-9d49-41db0dad0fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:21.757785 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1357/2697] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:21.765280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7fa5e-1a04-4f4b-b445-4c80a8588a17 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:21.765578 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7fa5e-1a04-4f4b-b445-4c80a8588a17 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:21.765864 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7fa5e-1a04-4f4b-b445-4c80a8588a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:21.785718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bcc7fa5e-1a04-4f4b-b445-4c80a8588a17 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:21.786964 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bcc7fa5e-1a04-4f4b-b445-4c80a8588a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:21.794933 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1341/2698] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:21.799551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02da96d-3eca-492f-9d49-41db0dad0fc7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:21.805779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:21.809898 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02da96d-3eca-492f-9d49-41db0dad0fc7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:21.816697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1a44d-067c-442f-9736-7a1312138020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:21.817366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1a44d-067c-442f-9736-7a1312138020 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:21.817749 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1a44d-067c-442f-9736-7a1312138020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:21.818261 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1a44d-067c-442f-9736-7a1312138020 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:21.819057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1a44d-067c-442f-9736-7a1312138020 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:21.819455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1a44d-067c-442f-9736-7a1312138020 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:21.821167 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1a44d-067c-442f-9736-7a1312138020 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:21.832711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1358/2699] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:21.836871 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7fa5e-1a04-4f4b-b445-4c80a8588a17 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:21.846076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7fa5e-1a04-4f4b-b445-4c80a8588a17 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:21.850808 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c638c9-a17d-47a1-8b96-67d717d30469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:21.851455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c638c9-a17d-47a1-8b96-67d717d30469 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:21.851788 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c638c9-a17d-47a1-8b96-67d717d30469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:21.852059 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c638c9-a17d-47a1-8b96-67d717d30469 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:21.852755 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c638c9-a17d-47a1-8b96-67d717d30469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:21.854668 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c638c9-a17d-47a1-8b96-67d717d30469 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:21.880111 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1342/2700] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:21 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:21.884075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1a44d-067c-442f-9736-7a1312138020 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:21.894230 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1a44d-067c-442f-9736-7a1312138020 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:21.951921 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431909d0-2628-4a4a-a501-409acd098643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:21.952534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431909d0-2628-4a4a-a501-409acd098643 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:21.952833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431909d0-2628-4a4a-a501-409acd098643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:21.953123 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431909d0-2628-4a4a-a501-409acd098643 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:21.953787 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431909d0-2628-4a4a-a501-409acd098643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:21.955729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431909d0-2628-4a4a-a501-409acd098643 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:21.974721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1359/2701] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:21 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:21.979461 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c638c9-a17d-47a1-8b96-67d717d30469 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:21.992121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c638c9-a17d-47a1-8b96-67d717d30469 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:21.996757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf141f4-a7c3-40f3-9306-453c70f29c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:21.997413 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf141f4-a7c3-40f3-9306-453c70f29c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:21.997697 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf141f4-a7c3-40f3-9306-453c70f29c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:21.997977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf141f4-a7c3-40f3-9306-453c70f29c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:21.998701 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf141f4-a7c3-40f3-9306-453c70f29c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:22.000335 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf141f4-a7c3-40f3-9306-453c70f29c3a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:22.012075 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1343/2702] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:22.015995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431909d0-2628-4a4a-a501-409acd098643 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:22.027141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431909d0-2628-4a4a-a501-409acd098643 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:22.031715 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76bc06-896b-43d6-a6a7-cee249dc6e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:22.032891 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76bc06-896b-43d6-a6a7-cee249dc6e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:22.033174 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76bc06-896b-43d6-a6a7-cee249dc6e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:22.033448 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76bc06-896b-43d6-a6a7-cee249dc6e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:22.034169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76bc06-896b-43d6-a6a7-cee249dc6e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:22.035771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76bc06-896b-43d6-a6a7-cee249dc6e92 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:22.045569 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1360/2703] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:22.050184 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf141f4-a7c3-40f3-9306-453c70f29c3a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:22.061214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf141f4-a7c3-40f3-9306-453c70f29c3a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:22.066198 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc18780-6b40-4d02-aeed-0c54b59e8a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:22.067072 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc18780-6b40-4d02-aeed-0c54b59e8a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:22.067354 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc18780-6b40-4d02-aeed-0c54b59e8a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:22.067626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc18780-6b40-4d02-aeed-0c54b59e8a6f admin admin] PATH_INFO: `/v3/projects/1c26b11f3e3340499a0855d9451dedfa/users/d595949faab3418d860ba694e1590e9f/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:22.083030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc18780-6b40-4d02-aeed-0c54b59e8a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c26b11f3e3340499a0855d9451dedfa, user_id=d595949faab3418d860ba694e1590e9f, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:22.085805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc18780-6b40-4d02-aeed-0c54b59e8a6f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:22.099992 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1344/2704] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:22 2022] PUT /identity/v3/projects/1c26b11f3e3340499a0855d9451dedfa/users/d595949faab3418d860ba694e1590e9f/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:22.111861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c62a7c-dbd9-45fe-a1ab-650f6c6093c4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:22.112158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c62a7c-dbd9-45fe-a1ab-650f6c6093c4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:22.112445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c62a7c-dbd9-45fe-a1ab-650f6c6093c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:22.138342 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f2c62a7c-dbd9-45fe-a1ab-650f6c6093c4 None None] MFA Rules not processed for user `d595949faab3418d860ba694e1590e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:22.139564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f2c62a7c-dbd9-45fe-a1ab-650f6c6093c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:22.201566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1361/2705] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:22.206005 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc18780-6b40-4d02-aeed-0c54b59e8a6f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:22.265651 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc18780-6b40-4d02-aeed-0c54b59e8a6f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:22.270514 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2f706-57d0-4365-a5e6-8735a239afa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:22.271162 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2f706-57d0-4365-a5e6-8735a239afa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:22.271444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2f706-57d0-4365-a5e6-8735a239afa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:22.271746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2f706-57d0-4365-a5e6-8735a239afa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:22.272425 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2f706-57d0-4365-a5e6-8735a239afa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:22.274302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2f706-57d0-4365-a5e6-8735a239afa5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:22.300101 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1345/2706] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:22 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:22.304208 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c62a7c-dbd9-45fe-a1ab-650f6c6093c4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:22.315279 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c62a7c-dbd9-45fe-a1ab-650f6c6093c4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:22.374321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa83a0f-9d08-49bf-acc4-2ba76afbb6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:22.375000 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa83a0f-9d08-49bf-acc4-2ba76afbb6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:22.375284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa83a0f-9d08-49bf-acc4-2ba76afbb6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:22.375615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa83a0f-9d08-49bf-acc4-2ba76afbb6a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:22.376300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa83a0f-9d08-49bf-acc4-2ba76afbb6a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:22.378209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa83a0f-9d08-49bf-acc4-2ba76afbb6a5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:22.385022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:22.401413 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1362/2707] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:22 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:22.405260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2f706-57d0-4365-a5e6-8735a239afa5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:22.417869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2f706-57d0-4365-a5e6-8735a239afa5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:22.422565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77d3b9-a8e8-41b9-8914-27a011b6a69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:22.423232 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77d3b9-a8e8-41b9-8914-27a011b6a69a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:22.423516 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77d3b9-a8e8-41b9-8914-27a011b6a69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:22.423842 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77d3b9-a8e8-41b9-8914-27a011b6a69a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:22.424607 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77d3b9-a8e8-41b9-8914-27a011b6a69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:22.426168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77d3b9-a8e8-41b9-8914-27a011b6a69a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:22.437299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1346/2708] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:22.441505 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa83a0f-9d08-49bf-acc4-2ba76afbb6a5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:22.452677 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa83a0f-9d08-49bf-acc4-2ba76afbb6a5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:22.458024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c73b8f-260d-43c8-ab02-c93348469371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:22.458879 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c73b8f-260d-43c8-ab02-c93348469371 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:22.461024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c73b8f-260d-43c8-ab02-c93348469371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:22.461184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c73b8f-260d-43c8-ab02-c93348469371 admin admin] PATH_INFO: `/v3/projects/f06ff56ef1f64c2ba71d89df681089e7/users/2f8431fcf5bf4c5a973c92b78ebad8ca/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:22.479117 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c73b8f-260d-43c8-ab02-c93348469371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f06ff56ef1f64c2ba71d89df681089e7, user_id=2f8431fcf5bf4c5a973c92b78ebad8ca, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:22.481863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c73b8f-260d-43c8-ab02-c93348469371 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:22.498991 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1363/2709] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:22 2022] PUT /identity/v3/projects/f06ff56ef1f64c2ba71d89df681089e7/users/2f8431fcf5bf4c5a973c92b78ebad8ca/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:22.512107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77d3b9-a8e8-41b9-8914-27a011b6a69a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:22.523114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77d3b9-a8e8-41b9-8914-27a011b6a69a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:22.537472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:22.591833 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8941c3-da28-4ca9-9e6b-fb96c99f7aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:22.592433 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8941c3-da28-4ca9-9e6b-fb96c99f7aae admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:22.592721 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8941c3-da28-4ca9-9e6b-fb96c99f7aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:22.593054 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8941c3-da28-4ca9-9e6b-fb96c99f7aae admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:22.593794 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8941c3-da28-4ca9-9e6b-fb96c99f7aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:22.596098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8941c3-da28-4ca9-9e6b-fb96c99f7aae admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:22.607491 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1347/2710] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:22.611346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c73b8f-260d-43c8-ab02-c93348469371 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:22.623217 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c73b8f-260d-43c8-ab02-c93348469371 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:22.628480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625e27d-0010-43a7-a436-6cbdc9f3dd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:22.628989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625e27d-0010-43a7-a436-6cbdc9f3dd77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:22.629270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625e27d-0010-43a7-a436-6cbdc9f3dd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:22.629546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625e27d-0010-43a7-a436-6cbdc9f3dd77 admin admin] PATH_INFO: `/v3/projects/f06ff56ef1f64c2ba71d89df681089e7/users/2f8431fcf5bf4c5a973c92b78ebad8ca/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:22.631504 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625e27d-0010-43a7-a436-6cbdc9f3dd77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f06ff56ef1f64c2ba71d89df681089e7, user_id=2f8431fcf5bf4c5a973c92b78ebad8ca, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:22.634145 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625e27d-0010-43a7-a436-6cbdc9f3dd77 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:22.649543 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1364/2711] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:22 2022] PUT /identity/v3/projects/f06ff56ef1f64c2ba71d89df681089e7/users/2f8431fcf5bf4c5a973c92b78ebad8ca/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:22.660640 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb766d9f-56fc-497c-98f3-ad347d2916be None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:22.660972 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb766d9f-56fc-497c-98f3-ad347d2916be None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:22.661380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb766d9f-56fc-497c-98f3-ad347d2916be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:22.692622 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cb766d9f-56fc-497c-98f3-ad347d2916be None None] MFA Rules not processed for user `2f8431fcf5bf4c5a973c92b78ebad8ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:22.694023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cb766d9f-56fc-497c-98f3-ad347d2916be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:22.781011 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1348/2712] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:22.792340 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625e27d-0010-43a7-a436-6cbdc9f3dd77 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:22.794178 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1625e27d-0010-43a7-a436-6cbdc9f3dd77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:22.852718 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625e27d-0010-43a7-a436-6cbdc9f3dd77 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:22.856473 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ea99c-9b01-411b-98cd-6a62c95567cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:22.857790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ea99c-9b01-411b-98cd-6a62c95567cf service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:22.857790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ea99c-9b01-411b-98cd-6a62c95567cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:22.857790 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ea99c-9b01-411b-98cd-6a62c95567cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:22.868380 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ea99c-9b01-411b-98cd-6a62c95567cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:22.869927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ea99c-9b01-411b-98cd-6a62c95567cf service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:22.880876 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1365/2713] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:23.250650 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb766d9f-56fc-497c-98f3-ad347d2916be None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:23.322586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb766d9f-56fc-497c-98f3-ad347d2916be None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:23.327307 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:23.328285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:23.329249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:23.329616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] PATH_INFO: `/v3/users/d595949faab3418d860ba694e1590e9f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:23.330996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d595949faab3418d860ba694e1590e9f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:23.333042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:23.379976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:23.386366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd595949faab3418d860ba694e1590e9f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:23.387717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd595949faab3418d860ba694e1590e9f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:23.396738 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd595949faab3418d860ba694e1590e9f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:23.410806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd595949faab3418d860ba694e1590e9f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:23.418027 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd595949faab3418d860ba694e1590e9f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:23.424309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1349/2714] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:23 2022] DELETE /identity/v3/users/d595949faab3418d860ba694e1590e9f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:23.428574 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ca16-40ad-4113-ac19-fafc0d2cfad5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:23.428865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ca16-40ad-4113-ac19-fafc0d2cfad5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:23.429259 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ca16-40ad-4113-ac19-fafc0d2cfad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:23.447130 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9215ca16-40ad-4113-ac19-fafc0d2cfad5 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:23.448549 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9215ca16-40ad-4113-ac19-fafc0d2cfad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:23.523647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1366/2715] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:23.535481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:23.537103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:23.598366 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681e30e-e2c9-4a74-b03f-7dcfa796653f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:23.602043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547d1da-9d8c-4056-ac20-38cd5ff46ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:23.602713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547d1da-9d8c-4056-ac20-38cd5ff46ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:23.603015 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547d1da-9d8c-4056-ac20-38cd5ff46ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:23.603297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547d1da-9d8c-4056-ac20-38cd5ff46ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:23.612693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547d1da-9d8c-4056-ac20-38cd5ff46ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:23.614126 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547d1da-9d8c-4056-ac20-38cd5ff46ca4 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:23.624229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1350/2716] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:23.913188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215ca16-40ad-4113-ac19-fafc0d2cfad5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:23.914666 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9215ca16-40ad-4113-ac19-fafc0d2cfad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:23.927994 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215ca16-40ad-4113-ac19-fafc0d2cfad5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:23.932753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:23.933626 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:23.933912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:23.934192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] PATH_INFO: `/v3/projects/1c26b11f3e3340499a0855d9451dedfa` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:23.935804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c26b11f3e3340499a0855d9451dedfa)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:23.938043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:23.989880 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c26b11f3e3340499a0855d9451dedfa', 'request_id': 'req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:23.991071 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:23.998169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c26b11f3e3340499a0855d9451dedfa', 'request_id': 'req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.007584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c26b11f3e3340499a0855d9451dedfa', 'request_id': 'req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.020752 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] The token cache is being invalidate because project 1c26b11f3e3340499a0855d9451dedfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:24.021318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.022068 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.023760 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1367/2717] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:23 2022] DELETE /identity/v3/projects/1c26b11f3e3340499a0855d9451dedfa => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:24.027679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547d1da-9d8c-4056-ac20-38cd5ff46ca4 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:24.037556 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547d1da-9d8c-4056-ac20-38cd5ff46ca4 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:24.101958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a260401-c8db-4e90-b022-8d970eb1b0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:24.101958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a260401-c8db-4e90-b022-8d970eb1b0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:24.101958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a260401-c8db-4e90-b022-8d970eb1b0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:24.101958 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a260401-c8db-4e90-b022-8d970eb1b0e0 admin admin] PATH_INFO: `/v3/users/2f8431fcf5bf4c5a973c92b78ebad8ca` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:24.102355 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a260401-c8db-4e90-b022-8d970eb1b0e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f8431fcf5bf4c5a973c92b78ebad8ca)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:24.102355 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a260401-c8db-4e90-b022-8d970eb1b0e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:24.150711 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5a260401-c8db-4e90-b022-8d970eb1b0e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f8431fcf5bf4c5a973c92b78ebad8ca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.151464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5a260401-c8db-4e90-b022-8d970eb1b0e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f8431fcf5bf4c5a973c92b78ebad8ca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.159002 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5a260401-c8db-4e90-b022-8d970eb1b0e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f8431fcf5bf4c5a973c92b78ebad8ca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.166695 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:24.174480 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5a260401-c8db-4e90-b022-8d970eb1b0e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f8431fcf5bf4c5a973c92b78ebad8ca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.181876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5a260401-c8db-4e90-b022-8d970eb1b0e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f8431fcf5bf4c5a973c92b78ebad8ca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.188082 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1351/2718] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:24 2022] DELETE /identity/v3/users/2f8431fcf5bf4c5a973c92b78ebad8ca => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:24.588924 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:24.590534 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:24.614553 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4eac3-1e6a-4c7e-8866-e9d96b326728 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:24.683895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4c1b7-da01-447b-8dee-815aac88aa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:24.684940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4c1b7-da01-447b-8dee-815aac88aa11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:24.685316 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4c1b7-da01-447b-8dee-815aac88aa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:24.685691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4c1b7-da01-447b-8dee-815aac88aa11 admin admin] PATH_INFO: `/v3/projects/f06ff56ef1f64c2ba71d89df681089e7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:24.686975 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4c1b7-da01-447b-8dee-815aac88aa11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f06ff56ef1f64c2ba71d89df681089e7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:24.689289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4c1b7-da01-447b-8dee-815aac88aa11 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:24.753172 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-16a4c1b7-da01-447b-8dee-815aac88aa11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f06ff56ef1f64c2ba71d89df681089e7', 'request_id': 'req-16a4c1b7-da01-447b-8dee-815aac88aa11'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.760374 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-16a4c1b7-da01-447b-8dee-815aac88aa11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f06ff56ef1f64c2ba71d89df681089e7', 'request_id': 'req-16a4c1b7-da01-447b-8dee-815aac88aa11'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.768634 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-16a4c1b7-da01-447b-8dee-815aac88aa11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f06ff56ef1f64c2ba71d89df681089e7', 'request_id': 'req-16a4c1b7-da01-447b-8dee-815aac88aa11'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.784656 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-16a4c1b7-da01-447b-8dee-815aac88aa11 admin admin] The token cache is being invalidate because project f06ff56ef1f64c2ba71d89df681089e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:24.784870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-16a4c1b7-da01-447b-8dee-815aac88aa11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.785572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-16a4c1b7-da01-447b-8dee-815aac88aa11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.787224 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1368/2719] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:24 2022] DELETE /identity/v3/projects/f06ff56ef1f64c2ba71d89df681089e7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:24.794519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa708834-568e-4821-9195-3281e60207e1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:24.794519 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa708834-568e-4821-9195-3281e60207e1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:24.795136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa708834-568e-4821-9195-3281e60207e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:24.819873 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-aa708834-568e-4821-9195-3281e60207e1 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:24.821565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aa708834-568e-4821-9195-3281e60207e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:24.894129 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1352/2720] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:24.956169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb212717-3d09-4fc4-b9af-2e0b597421aa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:24.956712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb212717-3d09-4fc4-b9af-2e0b597421aa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:24.956712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb212717-3d09-4fc4-b9af-2e0b597421aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:24.976923 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bb212717-3d09-4fc4-b9af-2e0b597421aa None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:24.978140 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bb212717-3d09-4fc4-b9af-2e0b597421aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:24.988260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1369/2721] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:24.992581 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa708834-568e-4821-9195-3281e60207e1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:25.002406 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa708834-568e-4821-9195-3281e60207e1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:25.007098 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a1099-8667-476d-b566-365b5b9032c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:25.007886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a1099-8667-476d-b566-365b5b9032c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.008242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a1099-8667-476d-b566-365b5b9032c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.008616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a1099-8667-476d-b566-365b5b9032c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.009357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a1099-8667-476d-b566-365b5b9032c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:25.009662 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a1099-8667-476d-b566-365b5b9032c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:25.011308 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a1099-8667-476d-b566-365b5b9032c1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:25.024118 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1353/2722] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:11:25.028571 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb212717-3d09-4fc4-b9af-2e0b597421aa None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:25.038442 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb212717-3d09-4fc4-b9af-2e0b597421aa None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:25.043238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ac73b-7d88-46af-8608-7d3118cc437d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:25.044091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ac73b-7d88-46af-8608-7d3118cc437d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.044383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ac73b-7d88-46af-8608-7d3118cc437d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.044662 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ac73b-7d88-46af-8608-7d3118cc437d admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.045395 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ac73b-7d88-46af-8608-7d3118cc437d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:25.047387 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ac73b-7d88-46af-8608-7d3118cc437d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:25.073230 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1370/2723] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:25 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:25.077577 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a1099-8667-476d-b566-365b5b9032c1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:25.089165 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a1099-8667-476d-b566-365b5b9032c1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:25.163861 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc8598-3a4f-4971-96f6-969d2500625f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:25.164679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc8598-3a4f-4971-96f6-969d2500625f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.165068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc8598-3a4f-4971-96f6-969d2500625f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.165403 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc8598-3a4f-4971-96f6-969d2500625f admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.166163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc8598-3a4f-4971-96f6-969d2500625f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:25.168324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc8598-3a4f-4971-96f6-969d2500625f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:25.194302 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1354/2724] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:25 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:25.199404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ac73b-7d88-46af-8608-7d3118cc437d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:25.214885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ac73b-7d88-46af-8608-7d3118cc437d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:25.222050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d85e9-0a2d-4524-9b3e-cac809f34e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:25.223001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d85e9-0a2d-4524-9b3e-cac809f34e52 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.223707 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d85e9-0a2d-4524-9b3e-cac809f34e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.224702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d85e9-0a2d-4524-9b3e-cac809f34e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.226248 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d85e9-0a2d-4524-9b3e-cac809f34e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:25.228335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d85e9-0a2d-4524-9b3e-cac809f34e52 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:25.240260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1371/2725] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:25.244330 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc8598-3a4f-4971-96f6-969d2500625f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:25.257718 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc8598-3a4f-4971-96f6-969d2500625f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:25.264848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ec246-2091-4531-970c-fd19274028ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:25.265625 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ec246-2091-4531-970c-fd19274028ca admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.266397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ec246-2091-4531-970c-fd19274028ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.266682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ec246-2091-4531-970c-fd19274028ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.268358 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ec246-2091-4531-970c-fd19274028ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:25.284999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ec246-2091-4531-970c-fd19274028ca admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:25.304675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1355/2726] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:25.305500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d85e9-0a2d-4524-9b3e-cac809f34e52 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:25.319488 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d85e9-0a2d-4524-9b3e-cac809f34e52 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:25.325060 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731050ac-069f-4dcf-bcd8-98d87de103f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:25.325961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731050ac-069f-4dcf-bcd8-98d87de103f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.326243 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731050ac-069f-4dcf-bcd8-98d87de103f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.326520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731050ac-069f-4dcf-bcd8-98d87de103f5 admin admin] PATH_INFO: `/v3/projects/b435b446ea544cd3b5c0f841955ad972/users/1b51733a7d9e4b568c6468ff5fb4c00c/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.353628 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731050ac-069f-4dcf-bcd8-98d87de103f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b435b446ea544cd3b5c0f841955ad972, user_id=1b51733a7d9e4b568c6468ff5fb4c00c, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:25.357460 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731050ac-069f-4dcf-bcd8-98d87de103f5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:25.377270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1372/2727] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:25 2022] PUT /identity/v3/projects/b435b446ea544cd3b5c0f841955ad972/users/1b51733a7d9e4b568c6468ff5fb4c00c/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:25.389049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18ca91-779c-4275-94f3-2068436c3543 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.389410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18ca91-779c-4275-94f3-2068436c3543 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.389661 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18ca91-779c-4275-94f3-2068436c3543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.426306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6e18ca91-779c-4275-94f3-2068436c3543 None None] MFA Rules not processed for user `1b51733a7d9e4b568c6468ff5fb4c00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:25.427608 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6e18ca91-779c-4275-94f3-2068436c3543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:25.473127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:25.506143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1356/2728] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:25.510741 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731050ac-069f-4dcf-bcd8-98d87de103f5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:25.517111 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:25.585804 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731050ac-069f-4dcf-bcd8-98d87de103f5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:25.591558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589476b-6770-48f9-898d-8bffb0cc90f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:25.592857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589476b-6770-48f9-898d-8bffb0cc90f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.592857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589476b-6770-48f9-898d-8bffb0cc90f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.592857 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589476b-6770-48f9-898d-8bffb0cc90f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.593503 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589476b-6770-48f9-898d-8bffb0cc90f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:25.596031 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589476b-6770-48f9-898d-8bffb0cc90f4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:25.626995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1373/2729] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:25 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:25.628817 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18ca91-779c-4275-94f3-2068436c3543 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:25.640854 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18ca91-779c-4275-94f3-2068436c3543 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:25.730327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c372ed-b6dc-4cd3-b53c-3efc44f254b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:25.730327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c372ed-b6dc-4cd3-b53c-3efc44f254b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.730327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c372ed-b6dc-4cd3-b53c-3efc44f254b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.730327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c372ed-b6dc-4cd3-b53c-3efc44f254b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.730978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c372ed-b6dc-4cd3-b53c-3efc44f254b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:25.730978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c372ed-b6dc-4cd3-b53c-3efc44f254b5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:25.750119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1357/2730] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:25 2022] POST /identity/v3/users => generated 297 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:25.753272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589476b-6770-48f9-898d-8bffb0cc90f4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:25.764386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589476b-6770-48f9-898d-8bffb0cc90f4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:25.769838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94165dd-3c39-4a55-acf4-48197bc4bb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:25.771370 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94165dd-3c39-4a55-acf4-48197bc4bb1d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.771370 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94165dd-3c39-4a55-acf4-48197bc4bb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.771370 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94165dd-3c39-4a55-acf4-48197bc4bb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.772168 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94165dd-3c39-4a55-acf4-48197bc4bb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:25.774020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94165dd-3c39-4a55-acf4-48197bc4bb1d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:25.786855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1374/2731] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:25.791268 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c372ed-b6dc-4cd3-b53c-3efc44f254b5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:25.801824 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c372ed-b6dc-4cd3-b53c-3efc44f254b5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:25.807542 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f87b23-fa4e-401c-b671-686845705808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:25.808857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f87b23-fa4e-401c-b671-686845705808 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.809145 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f87b23-fa4e-401c-b671-686845705808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.809426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f87b23-fa4e-401c-b671-686845705808 admin admin] PATH_INFO: `/v3/projects/4834324fd05d422ab74192ad78523bab/users/c66fbbb0c05b40bea6cc13d185c74b28/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.826822 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f87b23-fa4e-401c-b671-686845705808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4834324fd05d422ab74192ad78523bab, user_id=c66fbbb0c05b40bea6cc13d185c74b28, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:25.830597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f87b23-fa4e-401c-b671-686845705808 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:25.846157 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1358/2732] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:25 2022] PUT /identity/v3/projects/4834324fd05d422ab74192ad78523bab/users/c66fbbb0c05b40bea6cc13d185c74b28/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:25.850300 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94165dd-3c39-4a55-acf4-48197bc4bb1d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:25.861650 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94165dd-3c39-4a55-acf4-48197bc4bb1d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:25.945957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fba8ce-d679-4b3a-a74d-cfaec37c563c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:25.946806 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fba8ce-d679-4b3a-a74d-cfaec37c563c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.947176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fba8ce-d679-4b3a-a74d-cfaec37c563c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.947520 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fba8ce-d679-4b3a-a74d-cfaec37c563c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.948312 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fba8ce-d679-4b3a-a74d-cfaec37c563c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:25.950258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fba8ce-d679-4b3a-a74d-cfaec37c563c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:25.969858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1375/2733] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:25.973936 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f87b23-fa4e-401c-b671-686845705808 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:26.013902 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f87b23-fa4e-401c-b671-686845705808 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:26.020761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc20d86-8735-4df8-af28-f456b09edd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:26.022776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc20d86-8735-4df8-af28-f456b09edd9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.023164 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc20d86-8735-4df8-af28-f456b09edd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.023549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc20d86-8735-4df8-af28-f456b09edd9e admin admin] PATH_INFO: `/v3/projects/4834324fd05d422ab74192ad78523bab/users/c66fbbb0c05b40bea6cc13d185c74b28/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.026151 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc20d86-8735-4df8-af28-f456b09edd9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4834324fd05d422ab74192ad78523bab, user_id=c66fbbb0c05b40bea6cc13d185c74b28, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:26.030897 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc20d86-8735-4df8-af28-f456b09edd9e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:26.044485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1359/2734] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:25 2022] PUT /identity/v3/projects/4834324fd05d422ab74192ad78523bab/users/c66fbbb0c05b40bea6cc13d185c74b28/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:26.057364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995d92c-13e9-4057-8112-91eb15774c3c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.057653 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995d92c-13e9-4057-8112-91eb15774c3c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.057932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995d92c-13e9-4057-8112-91eb15774c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.090685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2995d92c-13e9-4057-8112-91eb15774c3c None None] MFA Rules not processed for user `c66fbbb0c05b40bea6cc13d185c74b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:26.090685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2995d92c-13e9-4057-8112-91eb15774c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:26.166702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:26.171090 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1376/2735] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:26.183900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc20d86-8735-4df8-af28-f456b09edd9e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:26.185580 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7cc20d86-8735-4df8-af28-f456b09edd9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:26.249069 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc20d86-8735-4df8-af28-f456b09edd9e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:26.253100 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbbfad-7710-47cd-9e3b-a230fd7ac19e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:26.253780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbbfad-7710-47cd-9e3b-a230fd7ac19e service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.254113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbbfad-7710-47cd-9e3b-a230fd7ac19e service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.254412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbbfad-7710-47cd-9e3b-a230fd7ac19e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.264978 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbbfad-7710-47cd-9e3b-a230fd7ac19e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:26.266950 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbbfad-7710-47cd-9e3b-a230fd7ac19e service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:26.277484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1360/2736] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:26.363193 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995d92c-13e9-4057-8112-91eb15774c3c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:26.372544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995d92c-13e9-4057-8112-91eb15774c3c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:26.376411 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a7730-e7e0-41e4-90f6-5cd9ecdb479d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:26.377191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a7730-e7e0-41e4-90f6-5cd9ecdb479d service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.377583 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a7730-e7e0-41e4-90f6-5cd9ecdb479d service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.377904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a7730-e7e0-41e4-90f6-5cd9ecdb479d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.439485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a7730-e7e0-41e4-90f6-5cd9ecdb479d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:26.440979 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a7730-e7e0-41e4-90f6-5cd9ecdb479d service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:26.451119 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1377/2737] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:26.709811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbbfad-7710-47cd-9e3b-a230fd7ac19e service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:26.744907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:26.775159 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbbfad-7710-47cd-9e3b-a230fd7ac19e service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:26.779730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:26.780591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.780869 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.781146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] PATH_INFO: `/v3/users/1b51733a7d9e4b568c6468ff5fb4c00c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.782863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b51733a7d9e4b568c6468ff5fb4c00c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:26.784867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:26.845767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b51733a7d9e4b568c6468ff5fb4c00c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:26.846570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b51733a7d9e4b568c6468ff5fb4c00c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:26.855814 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b51733a7d9e4b568c6468ff5fb4c00c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:26.873600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b51733a7d9e4b568c6468ff5fb4c00c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:26.881276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b51733a7d9e4b568c6468ff5fb4c00c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:26.888301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1361/2738] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:26 2022] DELETE /identity/v3/users/1b51733a7d9e4b568c6468ff5fb4c00c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:26.893075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966e54d-666e-4563-b399-5fee643a9fb2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.893364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966e54d-666e-4563-b399-5fee643a9fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.893639 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966e54d-666e-4563-b399-5fee643a9fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.911697 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b966e54d-666e-4563-b399-5fee643a9fb2 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:26.912938 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b966e54d-666e-4563-b399-5fee643a9fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:26.973900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1378/2739] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:26.984547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:26.986315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:27.049763 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c278321-bd2a-40c9-81d5-2deb838fd841 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:27.054201 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100a0e1-8b4e-4b9b-a0ca-a8cd654013d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:27.054860 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100a0e1-8b4e-4b9b-a0ca-a8cd654013d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:27.055142 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100a0e1-8b4e-4b9b-a0ca-a8cd654013d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:27.055418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100a0e1-8b4e-4b9b-a0ca-a8cd654013d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:27.070776 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100a0e1-8b4e-4b9b-a0ca-a8cd654013d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:27.072395 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100a0e1-8b4e-4b9b-a0ca-a8cd654013d9 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:27.083332 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1362/2740] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:27.397978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966e54d-666e-4563-b399-5fee643a9fb2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:27.399447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b966e54d-666e-4563-b399-5fee643a9fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:27.411823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966e54d-666e-4563-b399-5fee643a9fb2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:27.416623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:27.417507 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:27.417772 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:27.418111 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] PATH_INFO: `/v3/projects/b435b446ea544cd3b5c0f841955ad972` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:27.419232 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b435b446ea544cd3b5c0f841955ad972)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:27.421195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:27.474928 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b435b446ea544cd3b5c0f841955ad972', 'request_id': 'req-0a3d6928-dfad-4390-bc60-7973884bdd33'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:27.481304 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b435b446ea544cd3b5c0f841955ad972', 'request_id': 'req-0a3d6928-dfad-4390-bc60-7973884bdd33'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:27.487537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b435b446ea544cd3b5c0f841955ad972', 'request_id': 'req-0a3d6928-dfad-4390-bc60-7973884bdd33'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:27.499328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] The token cache is being invalidate because project b435b446ea544cd3b5c0f841955ad972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:27.499599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:27.500259 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:27.502306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1379/2741] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:27 2022] DELETE /identity/v3/projects/b435b446ea544cd3b5c0f841955ad972 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:27.506356 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100a0e1-8b4e-4b9b-a0ca-a8cd654013d9 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:27.516300 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100a0e1-8b4e-4b9b-a0ca-a8cd654013d9 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:27.565071 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:27.575551 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c322c2e-665d-4b66-a94d-1c9002e6f3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:27.576446 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c322c2e-665d-4b66-a94d-1c9002e6f3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:27.576726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c322c2e-665d-4b66-a94d-1c9002e6f3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:27.577001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c322c2e-665d-4b66-a94d-1c9002e6f3d5 admin admin] PATH_INFO: `/v3/users/c66fbbb0c05b40bea6cc13d185c74b28` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:27.578167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c322c2e-665d-4b66-a94d-1c9002e6f3d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c66fbbb0c05b40bea6cc13d185c74b28)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:27.580694 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c322c2e-665d-4b66-a94d-1c9002e6f3d5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:27.627409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0c322c2e-665d-4b66-a94d-1c9002e6f3d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c66fbbb0c05b40bea6cc13d185c74b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:27.628110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0c322c2e-665d-4b66-a94d-1c9002e6f3d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c66fbbb0c05b40bea6cc13d185c74b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:27.636225 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0c322c2e-665d-4b66-a94d-1c9002e6f3d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c66fbbb0c05b40bea6cc13d185c74b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:27.648437 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0c322c2e-665d-4b66-a94d-1c9002e6f3d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c66fbbb0c05b40bea6cc13d185c74b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:27.655031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0c322c2e-665d-4b66-a94d-1c9002e6f3d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c66fbbb0c05b40bea6cc13d185c74b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:27.660886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1363/2742] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:27 2022] DELETE /identity/v3/users/c66fbbb0c05b40bea6cc13d185c74b28 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:27.927437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:27.928937 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:27.942706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d6928-dfad-4390-bc60-7973884bdd33 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:27.962398 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:28.008154 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c6fb1-0a02-4175-b8d0-4ddd43549911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.009366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c6fb1-0a02-4175-b8d0-4ddd43549911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.009366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c6fb1-0a02-4175-b8d0-4ddd43549911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.009570 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c6fb1-0a02-4175-b8d0-4ddd43549911 admin admin] PATH_INFO: `/v3/projects/4834324fd05d422ab74192ad78523bab` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.010721 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c6fb1-0a02-4175-b8d0-4ddd43549911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4834324fd05d422ab74192ad78523bab)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.012771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c6fb1-0a02-4175-b8d0-4ddd43549911 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.060748 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f63c6fb1-0a02-4175-b8d0-4ddd43549911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4834324fd05d422ab74192ad78523bab', 'request_id': 'req-f63c6fb1-0a02-4175-b8d0-4ddd43549911'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.067085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f63c6fb1-0a02-4175-b8d0-4ddd43549911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4834324fd05d422ab74192ad78523bab', 'request_id': 'req-f63c6fb1-0a02-4175-b8d0-4ddd43549911'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.072916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f63c6fb1-0a02-4175-b8d0-4ddd43549911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4834324fd05d422ab74192ad78523bab', 'request_id': 'req-f63c6fb1-0a02-4175-b8d0-4ddd43549911'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.085786 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f63c6fb1-0a02-4175-b8d0-4ddd43549911 admin admin] The token cache is being invalidate because project 4834324fd05d422ab74192ad78523bab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:28.086075 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f63c6fb1-0a02-4175-b8d0-4ddd43549911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.086684 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f63c6fb1-0a02-4175-b8d0-4ddd43549911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.088260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1380/2743] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:27 2022] DELETE /identity/v3/projects/4834324fd05d422ab74192ad78523bab => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:28.093793 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d76d7-9161-412a-8032-43e6762b718c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.094290 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d76d7-9161-412a-8032-43e6762b718c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.095051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d76d7-9161-412a-8032-43e6762b718c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.114596 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0e3d76d7-9161-412a-8032-43e6762b718c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:28.115847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0e3d76d7-9161-412a-8032-43e6762b718c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:28.175928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1364/2744] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:28.183298 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a6925-271a-42fb-9d3d-e37f0138273e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.183733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a6925-271a-42fb-9d3d-e37f0138273e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.184076 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a6925-271a-42fb-9d3d-e37f0138273e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.202504 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f34a6925-271a-42fb-9d3d-e37f0138273e None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:28.203864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f34a6925-271a-42fb-9d3d-e37f0138273e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:28.215932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1381/2745] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:28.217143 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d76d7-9161-412a-8032-43e6762b718c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:28.227912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d76d7-9161-412a-8032-43e6762b718c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:28.235066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94371763-1e26-4d06-9b5c-276c530cb624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.235066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94371763-1e26-4d06-9b5c-276c530cb624 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.235066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94371763-1e26-4d06-9b5c-276c530cb624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.235066 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94371763-1e26-4d06-9b5c-276c530cb624 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.236018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94371763-1e26-4d06-9b5c-276c530cb624 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:28.236018 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94371763-1e26-4d06-9b5c-276c530cb624 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.236710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a6925-271a-42fb-9d3d-e37f0138273e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:28.237484 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94371763-1e26-4d06-9b5c-276c530cb624 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.238502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f34a6925-271a-42fb-9d3d-e37f0138273e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:28.250374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1365/2746] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:28.252341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a6925-271a-42fb-9d3d-e37f0138273e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:28.254675 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94371763-1e26-4d06-9b5c-276c530cb624 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:28.257073 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ce344-517f-4935-9ee3-ab9cd3fb108f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.257936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ce344-517f-4935-9ee3-ab9cd3fb108f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.258214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ce344-517f-4935-9ee3-ab9cd3fb108f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.258485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ce344-517f-4935-9ee3-ab9cd3fb108f admin admin] PATH_INFO: `/v3/users/cd1b6f431ef34605a1053cf87191186d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.259702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ce344-517f-4935-9ee3-ab9cd3fb108f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd1b6f431ef34605a1053cf87191186d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.261610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ce344-517f-4935-9ee3-ab9cd3fb108f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.264400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94371763-1e26-4d06-9b5c-276c530cb624 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:28.269028 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205a54e-66e0-4323-9cf0-fcd97c7bfcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.269698 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205a54e-66e0-4323-9cf0-fcd97c7bfcfd admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.269981 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205a54e-66e0-4323-9cf0-fcd97c7bfcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.270520 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205a54e-66e0-4323-9cf0-fcd97c7bfcfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.271259 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205a54e-66e0-4323-9cf0-fcd97c7bfcfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.273177 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205a54e-66e0-4323-9cf0-fcd97c7bfcfd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.304231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1366/2747] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:28 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:28.305463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205a54e-66e0-4323-9cf0-fcd97c7bfcfd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:28.310548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b35ce344-517f-4935-9ee3-ab9cd3fb108f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd1b6f431ef34605a1053cf87191186d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.311210 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b35ce344-517f-4935-9ee3-ab9cd3fb108f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd1b6f431ef34605a1053cf87191186d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.312114 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b35ce344-517f-4935-9ee3-ab9cd3fb108f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd1b6f431ef34605a1053cf87191186d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.325255 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b35ce344-517f-4935-9ee3-ab9cd3fb108f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd1b6f431ef34605a1053cf87191186d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.332632 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b35ce344-517f-4935-9ee3-ab9cd3fb108f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd1b6f431ef34605a1053cf87191186d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.346816 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1382/2748] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:28 2022] DELETE /identity/v3/users/cd1b6f431ef34605a1053cf87191186d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:28.348689 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:28.351343 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e243c7-72e8-4118-a61e-67740c5eceb0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.351638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e243c7-72e8-4118-a61e-67740c5eceb0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.352002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e243c7-72e8-4118-a61e-67740c5eceb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.370445 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-49e243c7-72e8-4118-a61e-67740c5eceb0 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:28.371665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-49e243c7-72e8-4118-a61e-67740c5eceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:28.373537 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205a54e-66e0-4323-9cf0-fcd97c7bfcfd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:28.384464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:28.432632 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed257ce-e87f-40d8-909f-fab47fe8d233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.433275 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed257ce-e87f-40d8-909f-fab47fe8d233 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.433502 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed257ce-e87f-40d8-909f-fab47fe8d233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.433778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed257ce-e87f-40d8-909f-fab47fe8d233 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.434477 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed257ce-e87f-40d8-909f-fab47fe8d233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.435329 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1383/2749] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:28.436568 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed257ce-e87f-40d8-909f-fab47fe8d233 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.445433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e243c7-72e8-4118-a61e-67740c5eceb0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:28.446776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-49e243c7-72e8-4118-a61e-67740c5eceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:28.457747 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1367/2750] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:28 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:28.461963 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed257ce-e87f-40d8-909f-fab47fe8d233 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:28.463611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fed257ce-e87f-40d8-909f-fab47fe8d233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:28.475063 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed257ce-e87f-40d8-909f-fab47fe8d233 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:28.480590 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960f46e-e46c-472d-a718-07047f8ddce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.481456 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960f46e-e46c-472d-a718-07047f8ddce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.481812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960f46e-e46c-472d-a718-07047f8ddce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.482375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960f46e-e46c-472d-a718-07047f8ddce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.483141 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960f46e-e46c-472d-a718-07047f8ddce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.484763 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960f46e-e46c-472d-a718-07047f8ddce5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.495593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1368/2751] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:28.499478 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960f46e-e46c-472d-a718-07047f8ddce5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:28.505051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e243c7-72e8-4118-a61e-67740c5eceb0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:28.508753 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54776975-ef72-480c-9f23-c90ef73f6db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.509397 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54776975-ef72-480c-9f23-c90ef73f6db7 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.509679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54776975-ef72-480c-9f23-c90ef73f6db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.509957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54776975-ef72-480c-9f23-c90ef73f6db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.510124 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960f46e-e46c-472d-a718-07047f8ddce5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:28.514899 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1de4-12cf-4bd2-b5fa-2bb247342f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.515535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1de4-12cf-4bd2-b5fa-2bb247342f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.515858 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1de4-12cf-4bd2-b5fa-2bb247342f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.516133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1de4-12cf-4bd2-b5fa-2bb247342f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.516924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c1de4-12cf-4bd2-b5fa-2bb247342f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.518492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c1de4-12cf-4bd2-b5fa-2bb247342f71 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.519043 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54776975-ef72-480c-9f23-c90ef73f6db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.520766 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54776975-ef72-480c-9f23-c90ef73f6db7 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.532696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1369/2752] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:28.532910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1384/2753] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:28.536626 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1de4-12cf-4bd2-b5fa-2bb247342f71 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:28.546554 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1de4-12cf-4bd2-b5fa-2bb247342f71 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:28.551671 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f7feb-f84e-4a54-a5a3-9d8b67bd45c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.552559 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f7feb-f84e-4a54-a5a3-9d8b67bd45c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.552838 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f7feb-f84e-4a54-a5a3-9d8b67bd45c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.553111 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f7feb-f84e-4a54-a5a3-9d8b67bd45c7 admin admin] PATH_INFO: `/v3/projects/6bbce5e597634a5689edef50f1d09fc9/users/91b84c02d377441b98c832626be47fa8/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.568415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f7feb-f84e-4a54-a5a3-9d8b67bd45c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bbce5e597634a5689edef50f1d09fc9, user_id=91b84c02d377441b98c832626be47fa8, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.571030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f7feb-f84e-4a54-a5a3-9d8b67bd45c7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.584036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1370/2754] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:28 2022] PUT /identity/v3/projects/6bbce5e597634a5689edef50f1d09fc9/users/91b84c02d377441b98c832626be47fa8/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:28.594679 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5d9d3-2656-4b12-b7c7-d79b59ef946b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.594978 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5d9d3-2656-4b12-b7c7-d79b59ef946b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.595260 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5d9d3-2656-4b12-b7c7-d79b59ef946b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.622284 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-59c5d9d3-2656-4b12-b7c7-d79b59ef946b None None] MFA Rules not processed for user `91b84c02d377441b98c832626be47fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:28.623600 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-59c5d9d3-2656-4b12-b7c7-d79b59ef946b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:28.696234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1385/2755] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:28.707547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f7feb-f84e-4a54-a5a3-9d8b67bd45c7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:28.709299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b08f7feb-f84e-4a54-a5a3-9d8b67bd45c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:28.768372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f7feb-f84e-4a54-a5a3-9d8b67bd45c7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:28.772289 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bccf6-6544-415e-af09-fa9db5a70b17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.772930 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bccf6-6544-415e-af09-fa9db5a70b17 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.773209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bccf6-6544-415e-af09-fa9db5a70b17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.773513 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bccf6-6544-415e-af09-fa9db5a70b17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.783121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bccf6-6544-415e-af09-fa9db5a70b17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.784601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bccf6-6544-415e-af09-fa9db5a70b17 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.794563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1371/2756] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:28.803647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5d9d3-2656-4b12-b7c7-d79b59ef946b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:28.805761 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-59c5d9d3-2656-4b12-b7c7-d79b59ef946b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:28.817246 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bccf6-6544-415e-af09-fa9db5a70b17 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:28.855196 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:28.864934 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:28.879095 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5d9d3-2656-4b12-b7c7-d79b59ef946b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:28.883706 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a5db3-8f2a-4b8f-8802-0c9136365ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.884568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a5db3-8f2a-4b8f-8802-0c9136365ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.884850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a5db3-8f2a-4b8f-8802-0c9136365ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.885128 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a5db3-8f2a-4b8f-8802-0c9136365ad5 admin admin] PATH_INFO: `/v3/projects/6bd70e3417224f9c8078be5b95c558a2` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.886306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a5db3-8f2a-4b8f-8802-0c9136365ad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd70e3417224f9c8078be5b95c558a2)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.887229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bccf6-6544-415e-af09-fa9db5a70b17 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:28.888265 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a5db3-8f2a-4b8f-8802-0c9136365ad5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.892342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e794be3-c885-4ac3-a228-e17cb0c0e606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.893212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e794be3-c885-4ac3-a228-e17cb0c0e606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.893495 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e794be3-c885-4ac3-a228-e17cb0c0e606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.893775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e794be3-c885-4ac3-a228-e17cb0c0e606 admin admin] PATH_INFO: `/v3/users/91b84c02d377441b98c832626be47fa8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.894977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e794be3-c885-4ac3-a228-e17cb0c0e606 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91b84c02d377441b98c832626be47fa8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.897239 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e794be3-c885-4ac3-a228-e17cb0c0e606 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.937420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-735a5db3-8f2a-4b8f-8802-0c9136365ad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd70e3417224f9c8078be5b95c558a2', 'request_id': 'req-735a5db3-8f2a-4b8f-8802-0c9136365ad5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.943429 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-735a5db3-8f2a-4b8f-8802-0c9136365ad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd70e3417224f9c8078be5b95c558a2', 'request_id': 'req-735a5db3-8f2a-4b8f-8802-0c9136365ad5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.945439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9e794be3-c885-4ac3-a228-e17cb0c0e606 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91b84c02d377441b98c832626be47fa8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.946164 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9e794be3-c885-4ac3-a228-e17cb0c0e606 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91b84c02d377441b98c832626be47fa8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.949403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-735a5db3-8f2a-4b8f-8802-0c9136365ad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd70e3417224f9c8078be5b95c558a2', 'request_id': 'req-735a5db3-8f2a-4b8f-8802-0c9136365ad5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.954677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9e794be3-c885-4ac3-a228-e17cb0c0e606 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91b84c02d377441b98c832626be47fa8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.962624 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-735a5db3-8f2a-4b8f-8802-0c9136365ad5 admin admin] The token cache is being invalidate because project 6bd70e3417224f9c8078be5b95c558a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:28.962913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-735a5db3-8f2a-4b8f-8802-0c9136365ad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.963564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-735a5db3-8f2a-4b8f-8802-0c9136365ad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.965173 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1386/2757] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:28 2022] DELETE /identity/v3/projects/6bd70e3417224f9c8078be5b95c558a2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:28.967558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9e794be3-c885-4ac3-a228-e17cb0c0e606 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91b84c02d377441b98c832626be47fa8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.970417 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbccfb-bd49-4753-92da-abce50674b4c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.970779 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbccfb-bd49-4753-92da-abce50674b4c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.971162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbccfb-bd49-4753-92da-abce50674b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.975049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9e794be3-c885-4ac3-a228-e17cb0c0e606 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91b84c02d377441b98c832626be47fa8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:28.981439 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1372/2758] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:28 2022] DELETE /identity/v3/users/91b84c02d377441b98c832626be47fa8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:28.987163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3f0fd-d41c-4549-b715-39669bbbb05b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.987612 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3f0fd-d41c-4549-b715-39669bbbb05b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.988239 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3f0fd-d41c-4549-b715-39669bbbb05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.990902 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fdbbccfb-bd49-4753-92da-abce50674b4c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:28.992166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fdbbccfb-bd49-4753-92da-abce50674b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:29.007251 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f9f3f0fd-d41c-4549-b715-39669bbbb05b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:29.008996 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f9f3f0fd-d41c-4549-b715-39669bbbb05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:29.052196 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1387/2759] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:29.059905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a081d4-c6d3-4976-aea1-804d09facebd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.060195 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a081d4-c6d3-4976-aea1-804d09facebd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.060475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a081d4-c6d3-4976-aea1-804d09facebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.077558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1373/2760] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:29.077856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-64a081d4-c6d3-4976-aea1-804d09facebd None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:29.079067 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64a081d4-c6d3-4976-aea1-804d09facebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:29.087421 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1388/2761] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:29.092009 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3f0fd-d41c-4549-b715-39669bbbb05b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.094378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a081d4-c6d3-4976-aea1-804d09facebd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.096812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64a081d4-c6d3-4976-aea1-804d09facebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:29.103767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3f0fd-d41c-4549-b715-39669bbbb05b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.108898 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfed50-0c0d-4346-8145-e017306dee98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.109712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfed50-0c0d-4346-8145-e017306dee98 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.110716 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfed50-0c0d-4346-8145-e017306dee98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.112001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfed50-0c0d-4346-8145-e017306dee98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.112756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfed50-0c0d-4346-8145-e017306dee98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:29.113067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfed50-0c0d-4346-8145-e017306dee98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.114726 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfed50-0c0d-4346-8145-e017306dee98 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.126920 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1374/2762] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:29.131060 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfed50-0c0d-4346-8145-e017306dee98 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.140847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfed50-0c0d-4346-8145-e017306dee98 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.146473 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e112a6-be25-4c04-8d88-2380de1468c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.147261 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e112a6-be25-4c04-8d88-2380de1468c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.147648 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e112a6-be25-4c04-8d88-2380de1468c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.148049 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e112a6-be25-4c04-8d88-2380de1468c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.148851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e112a6-be25-4c04-8d88-2380de1468c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.152487 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e112a6-be25-4c04-8d88-2380de1468c2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.159912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a081d4-c6d3-4976-aea1-804d09facebd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.164735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8944f-10f1-4176-8eb2-b55f58f8cb63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.165383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8944f-10f1-4176-8eb2-b55f58f8cb63 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.165665 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8944f-10f1-4176-8eb2-b55f58f8cb63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.165940 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8944f-10f1-4176-8eb2-b55f58f8cb63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.176983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8944f-10f1-4176-8eb2-b55f58f8cb63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.178425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8944f-10f1-4176-8eb2-b55f58f8cb63 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.180240 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1375/2763] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:29 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:29.184393 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e112a6-be25-4c04-8d88-2380de1468c2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.189205 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1389/2764] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:29.205691 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:29.260525 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e112a6-be25-4c04-8d88-2380de1468c2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.273870 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b10b49-df3e-407d-a812-c92656079d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.274672 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b10b49-df3e-407d-a812-c92656079d78 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.275021 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b10b49-df3e-407d-a812-c92656079d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.275369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b10b49-df3e-407d-a812-c92656079d78 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.276288 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b10b49-df3e-407d-a812-c92656079d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.278449 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b10b49-df3e-407d-a812-c92656079d78 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.299341 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1376/2765] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:29 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:29.303483 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8944f-10f1-4176-8eb2-b55f58f8cb63 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.313519 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8944f-10f1-4176-8eb2-b55f58f8cb63 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.318976 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa12d9a-e738-4a38-bc8e-ed36dca45323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.319896 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa12d9a-e738-4a38-bc8e-ed36dca45323 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.320181 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa12d9a-e738-4a38-bc8e-ed36dca45323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.320461 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa12d9a-e738-4a38-bc8e-ed36dca45323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.321191 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa12d9a-e738-4a38-bc8e-ed36dca45323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.322878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa12d9a-e738-4a38-bc8e-ed36dca45323 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.334076 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1390/2766] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:29.337750 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b10b49-df3e-407d-a812-c92656079d78 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.348693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b10b49-df3e-407d-a812-c92656079d78 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.353524 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2106165-c1ea-4406-95c0-73cb03e1f44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.354396 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2106165-c1ea-4406-95c0-73cb03e1f44e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.354678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2106165-c1ea-4406-95c0-73cb03e1f44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.354955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2106165-c1ea-4406-95c0-73cb03e1f44e admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.355722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2106165-c1ea-4406-95c0-73cb03e1f44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.357329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2106165-c1ea-4406-95c0-73cb03e1f44e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.369391 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1377/2767] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:29.373509 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa12d9a-e738-4a38-bc8e-ed36dca45323 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.383788 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa12d9a-e738-4a38-bc8e-ed36dca45323 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.388426 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046aee8-1e88-42c9-a979-6c93f50fa0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.389310 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046aee8-1e88-42c9-a979-6c93f50fa0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.389595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046aee8-1e88-42c9-a979-6c93f50fa0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.389878 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046aee8-1e88-42c9-a979-6c93f50fa0f9 admin admin] PATH_INFO: `/v3/projects/f211a72a8969467eb86d4fc5e28561e9/users/a3b017c5da2b4a66b180b30a8829d940/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.392873 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:29.409358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046aee8-1e88-42c9-a979-6c93f50fa0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f211a72a8969467eb86d4fc5e28561e9, user_id=a3b017c5da2b4a66b180b30a8829d940, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.412062 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046aee8-1e88-42c9-a979-6c93f50fa0f9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.425478 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1391/2768] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:29 2022] PUT /identity/v3/projects/f211a72a8969467eb86d4fc5e28561e9/users/a3b017c5da2b4a66b180b30a8829d940/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:29.435863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25cafa-b805-40ec-8cdb-12c1a64a1c7b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.436260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25cafa-b805-40ec-8cdb-12c1a64a1c7b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.436744 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25cafa-b805-40ec-8cdb-12c1a64a1c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.465385 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ab25cafa-b805-40ec-8cdb-12c1a64a1c7b None None] MFA Rules not processed for user `a3b017c5da2b4a66b180b30a8829d940`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:29.466599 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ab25cafa-b805-40ec-8cdb-12c1a64a1c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:29.483714 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046aee8-1e88-42c9-a979-6c93f50fa0f9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.485563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2046aee8-1e88-42c9-a979-6c93f50fa0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:29.544774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1378/2769] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:29.549105 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25cafa-b805-40ec-8cdb-12c1a64a1c7b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.555273 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046aee8-1e88-42c9-a979-6c93f50fa0f9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.559560 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25cafa-b805-40ec-8cdb-12c1a64a1c7b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.560139 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9f315-4c87-498e-9e4f-71f7cd441f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.561000 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9f315-4c87-498e-9e4f-71f7cd441f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.561288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9f315-4c87-498e-9e4f-71f7cd441f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.561568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9f315-4c87-498e-9e4f-71f7cd441f8d admin admin] PATH_INFO: `/v3/projects/6bbce5e597634a5689edef50f1d09fc9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.562757 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9f315-4c87-498e-9e4f-71f7cd441f8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bbce5e597634a5689edef50f1d09fc9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.564911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95b155-f84a-463f-b6df-c71323482ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.565219 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9f315-4c87-498e-9e4f-71f7cd441f8d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.565555 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95b155-f84a-463f-b6df-c71323482ded admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.565840 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95b155-f84a-463f-b6df-c71323482ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.566119 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95b155-f84a-463f-b6df-c71323482ded admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.566805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95b155-f84a-463f-b6df-c71323482ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.568753 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95b155-f84a-463f-b6df-c71323482ded admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.594109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1379/2770] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:29 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:29.597929 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95b155-f84a-463f-b6df-c71323482ded admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.614467 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-59f9f315-4c87-498e-9e4f-71f7cd441f8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bbce5e597634a5689edef50f1d09fc9', 'request_id': 'req-59f9f315-4c87-498e-9e4f-71f7cd441f8d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:29.620767 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-59f9f315-4c87-498e-9e4f-71f7cd441f8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bbce5e597634a5689edef50f1d09fc9', 'request_id': 'req-59f9f315-4c87-498e-9e4f-71f7cd441f8d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:29.627419 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-59f9f315-4c87-498e-9e4f-71f7cd441f8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bbce5e597634a5689edef50f1d09fc9', 'request_id': 'req-59f9f315-4c87-498e-9e4f-71f7cd441f8d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:29.640355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-59f9f315-4c87-498e-9e4f-71f7cd441f8d admin admin] The token cache is being invalidate because project 6bbce5e597634a5689edef50f1d09fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:29.640776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-59f9f315-4c87-498e-9e4f-71f7cd441f8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:29.642159 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-59f9f315-4c87-498e-9e4f-71f7cd441f8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:29.643815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1392/2771] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:29 2022] DELETE /identity/v3/projects/6bbce5e597634a5689edef50f1d09fc9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:29.648869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4ebcb-4d45-404a-921d-50773fd11c31 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.649092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4ebcb-4d45-404a-921d-50773fd11c31 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.649473 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4ebcb-4d45-404a-921d-50773fd11c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.661541 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95b155-f84a-463f-b6df-c71323482ded admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.666200 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:29.667391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-45e4ebcb-4d45-404a-921d-50773fd11c31 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:29.668712 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-45e4ebcb-4d45-404a-921d-50773fd11c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:29.721944 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fafdc-97ca-49cc-81bc-03b5357fb29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.722538 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fafdc-97ca-49cc-81bc-03b5357fb29f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.722820 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fafdc-97ca-49cc-81bc-03b5357fb29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.723097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fafdc-97ca-49cc-81bc-03b5357fb29f admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.723828 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fafdc-97ca-49cc-81bc-03b5357fb29f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.725748 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fafdc-97ca-49cc-81bc-03b5357fb29f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.728499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1393/2772] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:29.737042 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0956f1-70f5-438b-8415-6158a8359b02 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.737341 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0956f1-70f5-438b-8415-6158a8359b02 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.737620 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0956f1-70f5-438b-8415-6158a8359b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.749132 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1380/2773] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:29 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:29.750134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fafdc-97ca-49cc-81bc-03b5357fb29f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.752412 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2f5fafdc-97ca-49cc-81bc-03b5357fb29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:29.757865 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bf0956f1-70f5-438b-8415-6158a8359b02 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:29.759074 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bf0956f1-70f5-438b-8415-6158a8359b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:29.764550 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fafdc-97ca-49cc-81bc-03b5357fb29f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.767617 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1394/2774] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:29.772022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0956f1-70f5-438b-8415-6158a8359b02 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.772238 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4352be-6971-44b1-b5b6-e1d62a00ecec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.772927 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4352be-6971-44b1-b5b6-e1d62a00ecec admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.773216 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4352be-6971-44b1-b5b6-e1d62a00ecec admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.773496 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4352be-6971-44b1-b5b6-e1d62a00ecec admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.774224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4352be-6971-44b1-b5b6-e1d62a00ecec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.775874 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4352be-6971-44b1-b5b6-e1d62a00ecec admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.781202 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0956f1-70f5-438b-8415-6158a8359b02 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.785849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda5b25-f2a0-4de1-a849-6782d8099bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.786001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1381/2775] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:29.786491 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda5b25-f2a0-4de1-a849-6782d8099bbe admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.786776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda5b25-f2a0-4de1-a849-6782d8099bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.787055 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda5b25-f2a0-4de1-a849-6782d8099bbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.787944 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda5b25-f2a0-4de1-a849-6782d8099bbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:29.788258 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda5b25-f2a0-4de1-a849-6782d8099bbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.789797 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4352be-6971-44b1-b5b6-e1d62a00ecec admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.790106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda5b25-f2a0-4de1-a849-6782d8099bbe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.791696 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:29.801748 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4352be-6971-44b1-b5b6-e1d62a00ecec admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.804018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1395/2776] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:11:29.807885 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda5b25-f2a0-4de1-a849-6782d8099bbe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.810372 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecc4c3-abec-4451-967d-8dad27920906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.811765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecc4c3-abec-4451-967d-8dad27920906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.812229 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecc4c3-abec-4451-967d-8dad27920906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.812799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecc4c3-abec-4451-967d-8dad27920906 admin admin] PATH_INFO: `/v3/projects/33189b2a4caa45d99d754d9dab95e775/users/66ef593433f94dc6a03c73e2f46b9581/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.818856 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda5b25-f2a0-4de1-a849-6782d8099bbe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.824292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7268f7-4dde-45fb-9bea-4989dd0f746a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.824960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7268f7-4dde-45fb-9bea-4989dd0f746a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.825247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7268f7-4dde-45fb-9bea-4989dd0f746a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.825524 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7268f7-4dde-45fb-9bea-4989dd0f746a admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.826214 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7268f7-4dde-45fb-9bea-4989dd0f746a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.828148 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7268f7-4dde-45fb-9bea-4989dd0f746a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.832093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecc4c3-abec-4451-967d-8dad27920906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33189b2a4caa45d99d754d9dab95e775, user_id=66ef593433f94dc6a03c73e2f46b9581, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.834911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecc4c3-abec-4451-967d-8dad27920906 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.847990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1382/2777] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:29 2022] PUT /identity/v3/projects/33189b2a4caa45d99d754d9dab95e775/users/66ef593433f94dc6a03c73e2f46b9581/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:29.851976 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecc4c3-abec-4451-967d-8dad27920906 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.857545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1396/2778] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:29 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:29.862049 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7268f7-4dde-45fb-9bea-4989dd0f746a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.915291 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecc4c3-abec-4451-967d-8dad27920906 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.920192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0456e0-0dad-4820-8c55-4ee235dea70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.920859 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0456e0-0dad-4820-8c55-4ee235dea70d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.921144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0456e0-0dad-4820-8c55-4ee235dea70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.921423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0456e0-0dad-4820-8c55-4ee235dea70d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.922152 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0456e0-0dad-4820-8c55-4ee235dea70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.924318 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0456e0-0dad-4820-8c55-4ee235dea70d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.924850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7268f7-4dde-45fb-9bea-4989dd0f746a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.929630 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d64922-b5cd-4144-bada-91361e65347e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.930318 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d64922-b5cd-4144-bada-91361e65347e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.930605 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d64922-b5cd-4144-bada-91361e65347e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.930887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d64922-b5cd-4144-bada-91361e65347e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.931578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d64922-b5cd-4144-bada-91361e65347e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.933535 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d64922-b5cd-4144-bada-91361e65347e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.934364 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1383/2779] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:29.938973 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0456e0-0dad-4820-8c55-4ee235dea70d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.948544 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0456e0-0dad-4820-8c55-4ee235dea70d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.953343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611a449-e3e8-4eec-b34f-2f82a1f50d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:29.954233 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611a449-e3e8-4eec-b34f-2f82a1f50d52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.954522 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611a449-e3e8-4eec-b34f-2f82a1f50d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.954801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611a449-e3e8-4eec-b34f-2f82a1f50d52 admin admin] PATH_INFO: `/v3/projects/33189b2a4caa45d99d754d9dab95e775/users/66ef593433f94dc6a03c73e2f46b9581/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:29.958870 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1397/2780] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:29 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:29.959847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611a449-e3e8-4eec-b34f-2f82a1f50d52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33189b2a4caa45d99d754d9dab95e775, user_id=66ef593433f94dc6a03c73e2f46b9581, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:29.959976 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d64922-b5cd-4144-bada-91361e65347e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:29.962511 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611a449-e3e8-4eec-b34f-2f82a1f50d52 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:29.969511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d64922-b5cd-4144-bada-91361e65347e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:29.973237 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1384/2781] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:29 2022] PUT /identity/v3/projects/33189b2a4caa45d99d754d9dab95e775/users/66ef593433f94dc6a03c73e2f46b9581/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:29.985023 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fc923-7a0f-4c9a-8e8d-4af9d4ab615d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:29.985845 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fc923-7a0f-4c9a-8e8d-4af9d4ab615d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:29.986116 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fc923-7a0f-4c9a-8e8d-4af9d4ab615d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.013408 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-db4fc923-7a0f-4c9a-8e8d-4af9d4ab615d None None] MFA Rules not processed for user `66ef593433f94dc6a03c73e2f46b9581`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:30.014774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-db4fc923-7a0f-4c9a-8e8d-4af9d4ab615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:30.028912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f67eee-904a-48e7-99af-e6aac0c68707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:30.029611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f67eee-904a-48e7-99af-e6aac0c68707 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.029883 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f67eee-904a-48e7-99af-e6aac0c68707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.030162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f67eee-904a-48e7-99af-e6aac0c68707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.030927 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f67eee-904a-48e7-99af-e6aac0c68707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:30.032615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f67eee-904a-48e7-99af-e6aac0c68707 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:30.043685 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1398/2782] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:30.046611 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f67eee-904a-48e7-99af-e6aac0c68707 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:30.052702 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:30.057015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f67eee-904a-48e7-99af-e6aac0c68707 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:30.062738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e055bdf-61ae-435d-99d2-0c3273bc6ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:30.064048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e055bdf-61ae-435d-99d2-0c3273bc6ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.064328 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e055bdf-61ae-435d-99d2-0c3273bc6ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.064742 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e055bdf-61ae-435d-99d2-0c3273bc6ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.065465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e055bdf-61ae-435d-99d2-0c3273bc6ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:30.067034 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e055bdf-61ae-435d-99d2-0c3273bc6ba6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:30.077403 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1399/2783] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:30.081348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e055bdf-61ae-435d-99d2-0c3273bc6ba6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:30.092217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1385/2784] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:29 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:30.095508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e055bdf-61ae-435d-99d2-0c3273bc6ba6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:30.100828 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab538087-112b-414c-bedf-4f1b668373d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:30.101711 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab538087-112b-414c-bedf-4f1b668373d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.102007 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab538087-112b-414c-bedf-4f1b668373d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.102290 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab538087-112b-414c-bedf-4f1b668373d1 admin admin] PATH_INFO: `/v3/projects/e2b09458ea34420b89bc3ad3f7171dcc/users/bbd3d47d7ec04e51b880552f9728559a/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.103735 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fc923-7a0f-4c9a-8e8d-4af9d4ab615d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:30.105088 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-db4fc923-7a0f-4c9a-8e8d-4af9d4ab615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:30.119115 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab538087-112b-414c-bedf-4f1b668373d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b09458ea34420b89bc3ad3f7171dcc, user_id=bbd3d47d7ec04e51b880552f9728559a, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:30.121813 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab538087-112b-414c-bedf-4f1b668373d1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:30.136710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1400/2785] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:30 2022] PUT /identity/v3/projects/e2b09458ea34420b89bc3ad3f7171dcc/users/bbd3d47d7ec04e51b880552f9728559a/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:30.146771 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b2588-46e4-4db1-85b3-c4006f07dbc7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.147091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b2588-46e4-4db1-85b3-c4006f07dbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.147378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b2588-46e4-4db1-85b3-c4006f07dbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.164611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fc923-7a0f-4c9a-8e8d-4af9d4ab615d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:30.173693 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-838b2588-46e4-4db1-85b3-c4006f07dbc7 None None] MFA Rules not processed for user `bbd3d47d7ec04e51b880552f9728559a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:30.174913 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-838b2588-46e4-4db1-85b3-c4006f07dbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:30.184313 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:30.212622 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb43a69-18c0-4ae7-b455-18be884bb141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:30.213276 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb43a69-18c0-4ae7-b455-18be884bb141 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.213563 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb43a69-18c0-4ae7-b455-18be884bb141 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.213844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb43a69-18c0-4ae7-b455-18be884bb141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.243905 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1401/2786] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:30.248480 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b2588-46e4-4db1-85b3-c4006f07dbc7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:30.276609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb43a69-18c0-4ae7-b455-18be884bb141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:30.278085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb43a69-18c0-4ae7-b455-18be884bb141 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:30.289400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1386/2787] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:30 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:30.316907 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b2588-46e4-4db1-85b3-c4006f07dbc7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:30.321869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd8919-da98-4d82-92b5-b3240aee15d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:30.322544 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd8919-da98-4d82-92b5-b3240aee15d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.322994 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd8919-da98-4d82-92b5-b3240aee15d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.323118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd8919-da98-4d82-92b5-b3240aee15d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.323833 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd8919-da98-4d82-92b5-b3240aee15d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:30.325759 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd8919-da98-4d82-92b5-b3240aee15d0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:30.366092 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb43a69-18c0-4ae7-b455-18be884bb141 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:30.367830 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1402/2788] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:30 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:30.372784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd8919-da98-4d82-92b5-b3240aee15d0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:30.375420 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb43a69-18c0-4ae7-b455-18be884bb141 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:30.412409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:30.435512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91bbad0-9a95-4c07-89ba-2f72f448b2fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:30.436281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bbad0-9a95-4c07-89ba-2f72f448b2fc service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.436639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bbad0-9a95-4c07-89ba-2f72f448b2fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.436993 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bbad0-9a95-4c07-89ba-2f72f448b2fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.438926 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c91bbad0-9a95-4c07-89ba-2f72f448b2fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:30.444240 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd8919-da98-4d82-92b5-b3240aee15d0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:30.452598 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afc3ef-386f-4e28-9fc2-8075873431b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:30.453309 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afc3ef-386f-4e28-9fc2-8075873431b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.453595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afc3ef-386f-4e28-9fc2-8075873431b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.453904 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afc3ef-386f-4e28-9fc2-8075873431b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.454604 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afc3ef-386f-4e28-9fc2-8075873431b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:30.456579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afc3ef-386f-4e28-9fc2-8075873431b5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:30.479798 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1403/2789] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:30 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:30.480888 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afc3ef-386f-4e28-9fc2-8075873431b5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:30.492429 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afc3ef-386f-4e28-9fc2-8075873431b5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:30.497447 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05f449-5999-4773-b15e-f146b319592b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:30.498065 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05f449-5999-4773-b15e-f146b319592b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.498370 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05f449-5999-4773-b15e-f146b319592b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.498638 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05f449-5999-4773-b15e-f146b319592b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.499368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05f449-5999-4773-b15e-f146b319592b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:30.501019 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05f449-5999-4773-b15e-f146b319592b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:30.508369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91bbad0-9a95-4c07-89ba-2f72f448b2fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:30.509812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91bbad0-9a95-4c07-89ba-2f72f448b2fc service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:30.511472 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1404/2790] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:30.515405 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05f449-5999-4773-b15e-f146b319592b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:30.522654 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1387/2791] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:30.525381 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05f449-5999-4773-b15e-f146b319592b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:30.530244 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b90f11-c569-4f9c-8459-7f6e42371c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:30.531344 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b90f11-c569-4f9c-8459-7f6e42371c62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.531735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b90f11-c569-4f9c-8459-7f6e42371c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.532546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b90f11-c569-4f9c-8459-7f6e42371c62 admin admin] PATH_INFO: `/v3/projects/c533caa2c2514ecfaaf2bb8f89982645/users/6e6b1331a7734dd8838744faf2178b92/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.546145 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:30.551500 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b90f11-c569-4f9c-8459-7f6e42371c62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c533caa2c2514ecfaaf2bb8f89982645, user_id=6e6b1331a7734dd8838744faf2178b92, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:30.554267 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b90f11-c569-4f9c-8459-7f6e42371c62 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:30.566976 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1405/2792] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:30 2022] PUT /identity/v3/projects/c533caa2c2514ecfaaf2bb8f89982645/users/6e6b1331a7734dd8838744faf2178b92/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:30.570999 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91bbad0-9a95-4c07-89ba-2f72f448b2fc service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:30.582221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91bbad0-9a95-4c07-89ba-2f72f448b2fc service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:30.649990 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781b17a-9257-461e-8c57-ca69d47afcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:30.650815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781b17a-9257-461e-8c57-ca69d47afcb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.651097 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781b17a-9257-461e-8c57-ca69d47afcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.651369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781b17a-9257-461e-8c57-ca69d47afcb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.652159 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781b17a-9257-461e-8c57-ca69d47afcb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:30.657078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781b17a-9257-461e-8c57-ca69d47afcb3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:30.665572 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1388/2793] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:30.669499 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b90f11-c569-4f9c-8459-7f6e42371c62 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:30.679848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b90f11-c569-4f9c-8459-7f6e42371c62 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:30.684475 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b73f2-5ab4-44d0-a840-d1c8814adc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:30.685346 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b73f2-5ab4-44d0-a840-d1c8814adc6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.685632 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b73f2-5ab4-44d0-a840-d1c8814adc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.685909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b73f2-5ab4-44d0-a840-d1c8814adc6b admin admin] PATH_INFO: `/v3/projects/c533caa2c2514ecfaaf2bb8f89982645/users/6e6b1331a7734dd8838744faf2178b92/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.688007 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b73f2-5ab4-44d0-a840-d1c8814adc6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c533caa2c2514ecfaaf2bb8f89982645, user_id=6e6b1331a7734dd8838744faf2178b92, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:30.690917 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b73f2-5ab4-44d0-a840-d1c8814adc6b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:30.701431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1406/2794] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:30 2022] PUT /identity/v3/projects/c533caa2c2514ecfaaf2bb8f89982645/users/6e6b1331a7734dd8838744faf2178b92/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:30.712321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2af04-6cd9-4718-b499-616b93a7132a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.712609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2af04-6cd9-4718-b499-616b93a7132a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.712884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2af04-6cd9-4718-b499-616b93a7132a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.738857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8cf2af04-6cd9-4718-b499-616b93a7132a None None] MFA Rules not processed for user `6e6b1331a7734dd8838744faf2178b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:30.740093 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8cf2af04-6cd9-4718-b499-616b93a7132a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:30.805955 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1389/2795] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:30.816578 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b73f2-5ab4-44d0-a840-d1c8814adc6b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:30.870866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b73f2-5ab4-44d0-a840-d1c8814adc6b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:30.874529 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1af70-dad8-4e83-b9c8-5d79161747ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:30.875160 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1af70-dad8-4e83-b9c8-5d79161747ef service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:30.875444 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1af70-dad8-4e83-b9c8-5d79161747ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:30.875750 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1af70-dad8-4e83-b9c8-5d79161747ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:30.931058 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1af70-dad8-4e83-b9c8-5d79161747ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:30.932513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1af70-dad8-4e83-b9c8-5d79161747ef service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:30.941911 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1407/2796] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:33.452950 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf2af04-6cd9-4718-b499-616b93a7132a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:33.466962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:33.521743 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf2af04-6cd9-4718-b499-616b93a7132a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:33.526699 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:33.527582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:33.527900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:33.528191 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] PATH_INFO: `/v3/users/bbd3d47d7ec04e51b880552f9728559a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:33.529405 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbd3d47d7ec04e51b880552f9728559a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:33.531342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:33.579768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbd3d47d7ec04e51b880552f9728559a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:33.580427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbd3d47d7ec04e51b880552f9728559a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:33.588273 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbd3d47d7ec04e51b880552f9728559a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:33.601348 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbd3d47d7ec04e51b880552f9728559a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:33.608339 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbd3d47d7ec04e51b880552f9728559a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:33.615464 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1390/2797] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:33 2022] DELETE /identity/v3/users/bbd3d47d7ec04e51b880552f9728559a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:33.620238 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9376fa-bbe0-47d1-aea5-e685d528699b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:33.620723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9376fa-bbe0-47d1-aea5-e685d528699b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:33.620848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9376fa-bbe0-47d1-aea5-e685d528699b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:33.639391 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ed9376fa-bbe0-47d1-aea5-e685d528699b None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:33.640672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ed9376fa-bbe0-47d1-aea5-e685d528699b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:33.699775 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1408/2798] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:33.709427 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:33.710867 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:33.768267 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dca52-883e-457a-9f17-7d58232fd558 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:33.772168 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cc5cf-998a-4da8-b27b-fba44050b4f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:33.772812 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cc5cf-998a-4da8-b27b-fba44050b4f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:33.773096 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cc5cf-998a-4da8-b27b-fba44050b4f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:33.773375 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cc5cf-998a-4da8-b27b-fba44050b4f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:33.782352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cc5cf-998a-4da8-b27b-fba44050b4f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:33.783816 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cc5cf-998a-4da8-b27b-fba44050b4f1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:33.793514 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1391/2799] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:34.153531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9376fa-bbe0-47d1-aea5-e685d528699b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:34.154968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ed9376fa-bbe0-47d1-aea5-e685d528699b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:34.167256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9376fa-bbe0-47d1-aea5-e685d528699b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:34.171960 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:34.173462 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:34.173746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:34.174027 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] PATH_INFO: `/v3/projects/e2b09458ea34420b89bc3ad3f7171dcc` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:34.175220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b09458ea34420b89bc3ad3f7171dcc)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:34.177175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:34.208136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:34.225792 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b09458ea34420b89bc3ad3f7171dcc', 'request_id': 'req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.231882 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b09458ea34420b89bc3ad3f7171dcc', 'request_id': 'req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.237936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b09458ea34420b89bc3ad3f7171dcc', 'request_id': 'req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.249562 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] The token cache is being invalidate because project e2b09458ea34420b89bc3ad3f7171dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:34.249849 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.250466 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.252179 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1409/2800] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:34 2022] DELETE /identity/v3/projects/e2b09458ea34420b89bc3ad3f7171dcc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:34.256347 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cc5cf-998a-4da8-b27b-fba44050b4f1 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:34.266073 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cc5cf-998a-4da8-b27b-fba44050b4f1 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:34.299374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:34.324719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:34.325518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:34.325795 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:34.326068 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] PATH_INFO: `/v3/users/6e6b1331a7734dd8838744faf2178b92` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:34.327241 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6b1331a7734dd8838744faf2178b92)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:34.329188 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:34.375277 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6b1331a7734dd8838744faf2178b92'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.375950 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6b1331a7734dd8838744faf2178b92'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.383101 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6b1331a7734dd8838744faf2178b92'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.395217 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6b1331a7734dd8838744faf2178b92'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.402121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6b1331a7734dd8838744faf2178b92'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.408574 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1392/2801] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:34 2022] DELETE /identity/v3/users/6e6b1331a7734dd8838744faf2178b92 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:34.674965 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:34.677149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:34.692048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206c4e5-673b-49cb-89b9-0c4f5416ad9a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:34.755768 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1874309-85e3-41cf-9dee-e23f54244c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:34.756629 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1874309-85e3-41cf-9dee-e23f54244c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:34.756909 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1874309-85e3-41cf-9dee-e23f54244c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:34.757182 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1874309-85e3-41cf-9dee-e23f54244c6c admin admin] PATH_INFO: `/v3/projects/c533caa2c2514ecfaaf2bb8f89982645` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:34.758388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1874309-85e3-41cf-9dee-e23f54244c6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c533caa2c2514ecfaaf2bb8f89982645)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:34.760345 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1874309-85e3-41cf-9dee-e23f54244c6c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:34.808465 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1874309-85e3-41cf-9dee-e23f54244c6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c533caa2c2514ecfaaf2bb8f89982645', 'request_id': 'req-e1874309-85e3-41cf-9dee-e23f54244c6c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.814552 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1874309-85e3-41cf-9dee-e23f54244c6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c533caa2c2514ecfaaf2bb8f89982645', 'request_id': 'req-e1874309-85e3-41cf-9dee-e23f54244c6c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.820107 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1874309-85e3-41cf-9dee-e23f54244c6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c533caa2c2514ecfaaf2bb8f89982645', 'request_id': 'req-e1874309-85e3-41cf-9dee-e23f54244c6c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.832124 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1874309-85e3-41cf-9dee-e23f54244c6c admin admin] The token cache is being invalidate because project c533caa2c2514ecfaaf2bb8f89982645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:34.832408 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1874309-85e3-41cf-9dee-e23f54244c6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.833017 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1874309-85e3-41cf-9dee-e23f54244c6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.834561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1410/2802] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:34 2022] DELETE /identity/v3/projects/c533caa2c2514ecfaaf2bb8f89982645 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:40.664255 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:40.665719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:40.678941 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51f651-7d31-49bb-b578-bd16ca42c229 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:40.734162 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:40.735058 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:40.735342 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:40.735620 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] PATH_INFO: `/v3/users/a3b017c5da2b4a66b180b30a8829d940` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:40.737483 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b017c5da2b4a66b180b30a8829d940)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:40.739456 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:40.784617 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b017c5da2b4a66b180b30a8829d940'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.785260 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b017c5da2b4a66b180b30a8829d940'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.792182 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b017c5da2b4a66b180b30a8829d940'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.794137 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:40.805815 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b017c5da2b4a66b180b30a8829d940'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.812967 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b017c5da2b4a66b180b30a8829d940'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.819414 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1393/2803] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:40 2022] DELETE /identity/v3/users/a3b017c5da2b4a66b180b30a8829d940 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:40.823968 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571aef4-8598-46eb-a165-9d5fa56aebe3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:40.824428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571aef4-8598-46eb-a165-9d5fa56aebe3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:40.824564 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571aef4-8598-46eb-a165-9d5fa56aebe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:40.841957 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f571aef4-8598-46eb-a165-9d5fa56aebe3 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:40.843170 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f571aef4-8598-46eb-a165-9d5fa56aebe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:40.886092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:40.907129 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1411/2804] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:40.919006 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:40.920669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:40.982807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817ebf9-e145-40f9-ba25-672c91e12eba admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:40.987845 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c8cb9-f3e1-498e-88b5-56243239655f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:40.988496 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c8cb9-f3e1-498e-88b5-56243239655f service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:40.988775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c8cb9-f3e1-498e-88b5-56243239655f service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:40.989050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c8cb9-f3e1-498e-88b5-56243239655f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:40.998549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c8cb9-f3e1-498e-88b5-56243239655f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:41.000299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c8cb9-f3e1-498e-88b5-56243239655f service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:41.011028 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1394/2805] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:41.269823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571aef4-8598-46eb-a165-9d5fa56aebe3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:41.279824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571aef4-8598-46eb-a165-9d5fa56aebe3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:41.283130 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfbdda-67df-4008-af9a-f8acc9cfc400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:41.284377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfbdda-67df-4008-af9a-f8acc9cfc400 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:41.284661 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfbdda-67df-4008-af9a-f8acc9cfc400 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:41.284933 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfbdda-67df-4008-af9a-f8acc9cfc400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:41.286725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ffdfbdda-67df-4008-af9a-f8acc9cfc400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:41.329116 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c8cb9-f3e1-498e-88b5-56243239655f service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:41.330806 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-060c8cb9-f3e1-498e-88b5-56243239655f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:41.344117 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c8cb9-f3e1-498e-88b5-56243239655f service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:41.349811 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:41.350684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:41.350961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:41.351236 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] PATH_INFO: `/v3/projects/f211a72a8969467eb86d4fc5e28561e9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:41.352444 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f211a72a8969467eb86d4fc5e28561e9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:41.354389 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:41.359289 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfbdda-67df-4008-af9a-f8acc9cfc400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:41.362741 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfbdda-67df-4008-af9a-f8acc9cfc400 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:41.373323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1412/2806] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:41 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:41.405012 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f211a72a8969467eb86d4fc5e28561e9', 'request_id': 'req-3945e8eb-a27e-40e8-bf03-dfa691317c78'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.413638 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f211a72a8969467eb86d4fc5e28561e9', 'request_id': 'req-3945e8eb-a27e-40e8-bf03-dfa691317c78'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.422756 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f211a72a8969467eb86d4fc5e28561e9', 'request_id': 'req-3945e8eb-a27e-40e8-bf03-dfa691317c78'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.434934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] The token cache is being invalidate because project f211a72a8969467eb86d4fc5e28561e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:41.435202 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.435888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.437922 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1395/2807] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:41 2022] DELETE /identity/v3/projects/f211a72a8969467eb86d4fc5e28561e9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:41.442220 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfbdda-67df-4008-af9a-f8acc9cfc400 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:41.452992 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfbdda-67df-4008-af9a-f8acc9cfc400 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:41.515275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b646c7e-2a09-441e-91a7-1ecf18627a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:41.516183 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b646c7e-2a09-441e-91a7-1ecf18627a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:41.516464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b646c7e-2a09-441e-91a7-1ecf18627a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:41.516740 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b646c7e-2a09-441e-91a7-1ecf18627a28 admin admin] PATH_INFO: `/v3/users/66ef593433f94dc6a03c73e2f46b9581` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:41.518555 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b646c7e-2a09-441e-91a7-1ecf18627a28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ef593433f94dc6a03c73e2f46b9581)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:41.520455 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b646c7e-2a09-441e-91a7-1ecf18627a28 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:41.569956 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1b646c7e-2a09-441e-91a7-1ecf18627a28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ef593433f94dc6a03c73e2f46b9581'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.570619 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1b646c7e-2a09-441e-91a7-1ecf18627a28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ef593433f94dc6a03c73e2f46b9581'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.571252 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1b646c7e-2a09-441e-91a7-1ecf18627a28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ef593433f94dc6a03c73e2f46b9581'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.583899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1b646c7e-2a09-441e-91a7-1ecf18627a28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ef593433f94dc6a03c73e2f46b9581'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.590592 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1b646c7e-2a09-441e-91a7-1ecf18627a28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ef593433f94dc6a03c73e2f46b9581'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.601270 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:41.606599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1413/2808] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:41 2022] DELETE /identity/v3/users/66ef593433f94dc6a03c73e2f46b9581 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:41.998834 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.000465 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:42.017325 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945e8eb-a27e-40e8-bf03-dfa691317c78 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.090853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2472c-ac4f-4dc4-a95d-28251516d43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.090853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2472c-ac4f-4dc4-a95d-28251516d43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.090853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2472c-ac4f-4dc4-a95d-28251516d43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.090853 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2472c-ac4f-4dc4-a95d-28251516d43a admin admin] PATH_INFO: `/v3/projects/33189b2a4caa45d99d754d9dab95e775` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.091383 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2472c-ac4f-4dc4-a95d-28251516d43a admin admin] RBAC: Authorizing `identity:delete_project(project_id=33189b2a4caa45d99d754d9dab95e775)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.091383 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2472c-ac4f-4dc4-a95d-28251516d43a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.097920 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:42.140939 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ebb2472c-ac4f-4dc4-a95d-28251516d43a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33189b2a4caa45d99d754d9dab95e775', 'request_id': 'req-ebb2472c-ac4f-4dc4-a95d-28251516d43a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:42.149586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ebb2472c-ac4f-4dc4-a95d-28251516d43a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33189b2a4caa45d99d754d9dab95e775', 'request_id': 'req-ebb2472c-ac4f-4dc4-a95d-28251516d43a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:42.155722 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ebb2472c-ac4f-4dc4-a95d-28251516d43a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33189b2a4caa45d99d754d9dab95e775', 'request_id': 'req-ebb2472c-ac4f-4dc4-a95d-28251516d43a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:42.166730 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ebb2472c-ac4f-4dc4-a95d-28251516d43a admin admin] The token cache is being invalidate because project 33189b2a4caa45d99d754d9dab95e775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:42.167099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ebb2472c-ac4f-4dc4-a95d-28251516d43a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:42.168041 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ebb2472c-ac4f-4dc4-a95d-28251516d43a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:42.169643 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1396/2809] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:41 2022] DELETE /identity/v3/projects/33189b2a4caa45d99d754d9dab95e775 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:42.174660 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844125db-4bf0-4b9a-ad19-96311e7cd197 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.175207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844125db-4bf0-4b9a-ad19-96311e7cd197 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.175207 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844125db-4bf0-4b9a-ad19-96311e7cd197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.194596 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-844125db-4bf0-4b9a-ad19-96311e7cd197 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:42.195900 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-844125db-4bf0-4b9a-ad19-96311e7cd197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:42.257863 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1414/2810] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:42.265591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3e2a6-4460-4e2e-bd99-e33f7be6322f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.265886 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3e2a6-4460-4e2e-bd99-e33f7be6322f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.266167 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3e2a6-4460-4e2e-bd99-e33f7be6322f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.285829 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3ee3e2a6-4460-4e2e-bd99-e33f7be6322f None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:42.287016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3ee3e2a6-4460-4e2e-bd99-e33f7be6322f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:42.295353 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1397/2811] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:42.299819 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844125db-4bf0-4b9a-ad19-96311e7cd197 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.309838 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844125db-4bf0-4b9a-ad19-96311e7cd197 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.314431 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705f4a1-3ff7-45ad-ab6a-9451742c2a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.315085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705f4a1-3ff7-45ad-ab6a-9451742c2a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.315379 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705f4a1-3ff7-45ad-ab6a-9451742c2a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.315663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705f4a1-3ff7-45ad-ab6a-9451742c2a2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.316404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705f4a1-3ff7-45ad-ab6a-9451742c2a2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:42.316708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705f4a1-3ff7-45ad-ab6a-9451742c2a2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.318297 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705f4a1-3ff7-45ad-ab6a-9451742c2a2d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.333267 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1415/2812] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:11:42.334964 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3e2a6-4460-4e2e-bd99-e33f7be6322f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.344349 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3e2a6-4460-4e2e-bd99-e33f7be6322f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.350118 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0653da7-2a21-4d7d-acaf-df83c9bfc3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.350791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0653da7-2a21-4d7d-acaf-df83c9bfc3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.351070 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0653da7-2a21-4d7d-acaf-df83c9bfc3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.351343 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0653da7-2a21-4d7d-acaf-df83c9bfc3e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.352099 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0653da7-2a21-4d7d-acaf-df83c9bfc3e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.354067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0653da7-2a21-4d7d-acaf-df83c9bfc3e1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.389369 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1398/2813] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:42 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:42.390434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705f4a1-3ff7-45ad-ab6a-9451742c2a2d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.400993 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705f4a1-3ff7-45ad-ab6a-9451742c2a2d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.458624 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37482e5f-850d-40da-895d-1fd526889e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.459213 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37482e5f-850d-40da-895d-1fd526889e63 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.459493 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37482e5f-850d-40da-895d-1fd526889e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.459802 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37482e5f-850d-40da-895d-1fd526889e63 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.460502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37482e5f-850d-40da-895d-1fd526889e63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.462407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37482e5f-850d-40da-895d-1fd526889e63 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.484990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1416/2814] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:42 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:42.489165 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0653da7-2a21-4d7d-acaf-df83c9bfc3e1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.499768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0653da7-2a21-4d7d-acaf-df83c9bfc3e1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.504411 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad509f06-011c-4eb2-8f93-8f301753ae00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.505224 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad509f06-011c-4eb2-8f93-8f301753ae00 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.505509 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad509f06-011c-4eb2-8f93-8f301753ae00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.505781 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad509f06-011c-4eb2-8f93-8f301753ae00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.506777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad509f06-011c-4eb2-8f93-8f301753ae00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.508401 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad509f06-011c-4eb2-8f93-8f301753ae00 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.519696 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1399/2815] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:42.524394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37482e5f-850d-40da-895d-1fd526889e63 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.535895 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37482e5f-850d-40da-895d-1fd526889e63 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.540698 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5397cd4-fd75-4123-a5e8-e94b8fbb8386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.541366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5397cd4-fd75-4123-a5e8-e94b8fbb8386 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.541653 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5397cd4-fd75-4123-a5e8-e94b8fbb8386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.541932 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5397cd4-fd75-4123-a5e8-e94b8fbb8386 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.542672 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5397cd4-fd75-4123-a5e8-e94b8fbb8386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.544459 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5397cd4-fd75-4123-a5e8-e94b8fbb8386 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.558864 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1417/2816] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:42.560230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad509f06-011c-4eb2-8f93-8f301753ae00 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.571223 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad509f06-011c-4eb2-8f93-8f301753ae00 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.576254 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780dfc8-238a-4b76-9e5c-e85c8d33bfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.577225 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780dfc8-238a-4b76-9e5c-e85c8d33bfc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.577576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780dfc8-238a-4b76-9e5c-e85c8d33bfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.577924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780dfc8-238a-4b76-9e5c-e85c8d33bfc5 admin admin] PATH_INFO: `/v3/projects/a11b1f24e26744faa5878a8e54caac2e/users/ad52212c31034103b96a7769cb3a8f13/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.593510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780dfc8-238a-4b76-9e5c-e85c8d33bfc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11b1f24e26744faa5878a8e54caac2e, user_id=ad52212c31034103b96a7769cb3a8f13, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.596888 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780dfc8-238a-4b76-9e5c-e85c8d33bfc5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.610816 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1400/2817] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:42 2022] PUT /identity/v3/projects/a11b1f24e26744faa5878a8e54caac2e/users/ad52212c31034103b96a7769cb3a8f13/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:42.622920 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473f1c3-f9cc-4657-8eb7-059c24e77071 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.623303 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473f1c3-f9cc-4657-8eb7-059c24e77071 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.623671 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473f1c3-f9cc-4657-8eb7-059c24e77071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.651010 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1473f1c3-f9cc-4657-8eb7-059c24e77071 None None] MFA Rules not processed for user `ad52212c31034103b96a7769cb3a8f13`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:42.652257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1473f1c3-f9cc-4657-8eb7-059c24e77071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:42.708132 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:42.718321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1418/2818] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:42.730134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780dfc8-238a-4b76-9e5c-e85c8d33bfc5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.731892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c780dfc8-238a-4b76-9e5c-e85c8d33bfc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:42.792144 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780dfc8-238a-4b76-9e5c-e85c8d33bfc5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.796059 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61859510-0dfd-48f7-b32c-d6965b536e83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.796758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61859510-0dfd-48f7-b32c-d6965b536e83 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.797038 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61859510-0dfd-48f7-b32c-d6965b536e83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.797312 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61859510-0dfd-48f7-b32c-d6965b536e83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.807306 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61859510-0dfd-48f7-b32c-d6965b536e83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.808802 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61859510-0dfd-48f7-b32c-d6965b536e83 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.819245 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1401/2819] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:45.392059 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473f1c3-f9cc-4657-8eb7-059c24e77071 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:45.453095 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473f1c3-f9cc-4657-8eb7-059c24e77071 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:45.457591 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:45.458448 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:45.458728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:45.459002 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] PATH_INFO: `/v3/users/ad52212c31034103b96a7769cb3a8f13` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:45.460198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad52212c31034103b96a7769cb3a8f13)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:45.462111 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:45.507070 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad52212c31034103b96a7769cb3a8f13'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:45.507735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad52212c31034103b96a7769cb3a8f13'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:45.508335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad52212c31034103b96a7769cb3a8f13'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:45.520842 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad52212c31034103b96a7769cb3a8f13'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:45.528239 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad52212c31034103b96a7769cb3a8f13'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:45.542784 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1419/2820] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:45 2022] DELETE /identity/v3/users/ad52212c31034103b96a7769cb3a8f13 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:45.547163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3580c8-f6a3-4a6c-af6f-733a8769907a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:45.547752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3580c8-f6a3-4a6c-af6f-733a8769907a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:45.547752 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3580c8-f6a3-4a6c-af6f-733a8769907a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:45.566283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cc3580c8-f6a3-4a6c-af6f-733a8769907a None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:45.567741 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cc3580c8-f6a3-4a6c-af6f-733a8769907a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:45.591281 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:45.628591 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1402/2821] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:45.638008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:45.639464 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:45.696269 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f8c96-68f2-49b3-b6bc-f48aa7e887a0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:45.699910 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a8cd7-f93a-4246-9920-34650f43310f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:45.700545 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a8cd7-f93a-4246-9920-34650f43310f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:45.700826 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a8cd7-f93a-4246-9920-34650f43310f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:45.701126 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a8cd7-f93a-4246-9920-34650f43310f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:45.710831 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a8cd7-f93a-4246-9920-34650f43310f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:45.712334 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a8cd7-f93a-4246-9920-34650f43310f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:45.721887 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1420/2822] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:45.992315 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3580c8-f6a3-4a6c-af6f-733a8769907a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:45.993848 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cc3580c8-f6a3-4a6c-af6f-733a8769907a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:46.005856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3580c8-f6a3-4a6c-af6f-733a8769907a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:46.010383 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:46.011248 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:46.011530 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:46.011847 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6 admin admin] PATH_INFO: `/v3/projects/a11b1f24e26744faa5878a8e54caac2e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:46.013037 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a11b1f24e26744faa5878a8e54caac2e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:46.014943 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:46.061777 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a11b1f24e26744faa5878a8e54caac2e', 'request_id': 'req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:46.069172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a11b1f24e26744faa5878a8e54caac2e', 'request_id': 'req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:46.074728 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a11b1f24e26744faa5878a8e54caac2e', 'request_id': 'req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:46.086030 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6 admin admin] The token cache is being invalidate because project a11b1f24e26744faa5878a8e54caac2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:46.086329 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:46.087039 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e6606d54-ca1a-47d9-8af0-27136fe0e8b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:46.088669 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1403/2823] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:45 2022] DELETE /identity/v3/projects/a11b1f24e26744faa5878a8e54caac2e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:46.093846 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70968e-ab6f-45b0-b496-b2fd75d41474 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:46.094162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70968e-ab6f-45b0-b496-b2fd75d41474 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:46.094440 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70968e-ab6f-45b0-b496-b2fd75d41474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:46.112155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8b70968e-ab6f-45b0-b496-b2fd75d41474 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:46.113331 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8b70968e-ab6f-45b0-b496-b2fd75d41474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:46.124976 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:46.175218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1421/2824] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:46.183109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b1acd-b1c9-4e69-ac0e-04e97ee67580 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:46.183404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b1acd-b1c9-4e69-ac0e-04e97ee67580 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:46.183712 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b1acd-b1c9-4e69-ac0e-04e97ee67580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:46.202070 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-374b1acd-b1c9-4e69-ac0e-04e97ee67580 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:46.203269 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-374b1acd-b1c9-4e69-ac0e-04e97ee67580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:46.211611 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1404/2825] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:46.215808 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70968e-ab6f-45b0-b496-b2fd75d41474 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:46.225361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70968e-ab6f-45b0-b496-b2fd75d41474 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:46.229930 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c53e7-4981-4bbe-8200-2d5d4b1352b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:46.230581 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c53e7-4981-4bbe-8200-2d5d4b1352b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:46.230869 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c53e7-4981-4bbe-8200-2d5d4b1352b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:46.231149 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c53e7-4981-4bbe-8200-2d5d4b1352b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:46.231908 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c53e7-4981-4bbe-8200-2d5d4b1352b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:46.232221 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c53e7-4981-4bbe-8200-2d5d4b1352b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:46.233876 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c53e7-4981-4bbe-8200-2d5d4b1352b5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:46.245961 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1422/2826] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:46.249961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b1acd-b1c9-4e69-ac0e-04e97ee67580 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:46.259765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b1acd-b1c9-4e69-ac0e-04e97ee67580 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:46.264265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee664c38-d1ae-47bc-8263-454efd725ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:46.264929 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee664c38-d1ae-47bc-8263-454efd725ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:46.265235 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee664c38-d1ae-47bc-8263-454efd725ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:46.265503 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee664c38-d1ae-47bc-8263-454efd725ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:46.266220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee664c38-d1ae-47bc-8263-454efd725ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:46.268190 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee664c38-d1ae-47bc-8263-454efd725ccd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:46.293357 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1405/2827] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:46 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:46.297429 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c53e7-4981-4bbe-8200-2d5d4b1352b5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:46.308026 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c53e7-4981-4bbe-8200-2d5d4b1352b5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:46.364812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491f6b0-ea88-45b6-87a5-5636843c7119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:46.365508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491f6b0-ea88-45b6-87a5-5636843c7119 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:46.365795 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491f6b0-ea88-45b6-87a5-5636843c7119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:46.366076 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491f6b0-ea88-45b6-87a5-5636843c7119 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:46.367425 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491f6b0-ea88-45b6-87a5-5636843c7119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:46.369387 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491f6b0-ea88-45b6-87a5-5636843c7119 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:46.388710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1423/2828] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:46 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:46.392872 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee664c38-d1ae-47bc-8263-454efd725ccd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:46.404031 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee664c38-d1ae-47bc-8263-454efd725ccd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:46.408561 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36128538-7852-44ff-b163-3d862199616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:46.409713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36128538-7852-44ff-b163-3d862199616c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:46.409989 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36128538-7852-44ff-b163-3d862199616c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:46.410263 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36128538-7852-44ff-b163-3d862199616c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:46.410986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36128538-7852-44ff-b163-3d862199616c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:46.412593 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36128538-7852-44ff-b163-3d862199616c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:46.422693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1406/2829] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:46.427179 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491f6b0-ea88-45b6-87a5-5636843c7119 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:46.437292 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491f6b0-ea88-45b6-87a5-5636843c7119 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:46.441915 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74002a24-ee24-4b37-9b78-7caa8bb54bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:46.442558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74002a24-ee24-4b37-9b78-7caa8bb54bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:46.442840 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74002a24-ee24-4b37-9b78-7caa8bb54bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:46.443110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74002a24-ee24-4b37-9b78-7caa8bb54bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:46.443861 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74002a24-ee24-4b37-9b78-7caa8bb54bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:46.445436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74002a24-ee24-4b37-9b78-7caa8bb54bcd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:46.455105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1424/2830] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:46.459575 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36128538-7852-44ff-b163-3d862199616c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:46.469423 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36128538-7852-44ff-b163-3d862199616c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:46.474547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5996ff4-6241-4049-9bc4-d31f588a8dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:46.475409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5996ff4-6241-4049-9bc4-d31f588a8dba admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:46.475719 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5996ff4-6241-4049-9bc4-d31f588a8dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:46.475997 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5996ff4-6241-4049-9bc4-d31f588a8dba admin admin] PATH_INFO: `/v3/projects/f5c3243de4c94fe99d6dd7f96c814e62/users/851f30b7cc36417986029dfcc40b124f/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:46.490496 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5996ff4-6241-4049-9bc4-d31f588a8dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c3243de4c94fe99d6dd7f96c814e62, user_id=851f30b7cc36417986029dfcc40b124f, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:46.493163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5996ff4-6241-4049-9bc4-d31f588a8dba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:46.506746 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1407/2831] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:46 2022] PUT /identity/v3/projects/f5c3243de4c94fe99d6dd7f96c814e62/users/851f30b7cc36417986029dfcc40b124f/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:46.517092 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8de0f7-91db-48f3-9823-c602222d4b6c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:46.517383 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8de0f7-91db-48f3-9823-c602222d4b6c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:46.517663 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8de0f7-91db-48f3-9823-c602222d4b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:46.544335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-af8de0f7-91db-48f3-9823-c602222d4b6c None None] MFA Rules not processed for user `851f30b7cc36417986029dfcc40b124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:46.545565 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af8de0f7-91db-48f3-9823-c602222d4b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:46.609045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1425/2832] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:46.619938 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5996ff4-6241-4049-9bc4-d31f588a8dba admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:46.621609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d5996ff4-6241-4049-9bc4-d31f588a8dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:46.675050 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:46.679189 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5996ff4-6241-4049-9bc4-d31f588a8dba admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:46.683384 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc15ba3-861b-4ede-b963-fab1442dc7a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3663612647934677b83fb42aaf06d6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:46.684137 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc15ba3-861b-4ede-b963-fab1442dc7a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:46.684431 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc15ba3-861b-4ede-b963-fab1442dc7a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:46.685114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc15ba3-861b-4ede-b963-fab1442dc7a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:46.694448 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc15ba3-861b-4ede-b963-fab1442dc7a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:46.695932 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc15ba3-861b-4ede-b963-fab1442dc7a2 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:46.704801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1408/2833] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:48.043368 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa077d-a78f-495f-b84f-50259c71f9fe None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:48.043936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa077d-a78f-495f-b84f-50259c71f9fe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:48.043936 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa077d-a78f-495f-b84f-50259c71f9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:48.062218 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-beaa077d-a78f-495f-b84f-50259c71f9fe None None] MFA Rules not processed for user `851f30b7cc36417986029dfcc40b124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:48.063409 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-beaa077d-a78f-495f-b84f-50259c71f9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:48.070724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1426/2834] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:11:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:56.700715 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:56.865613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc15ba3-861b-4ede-b963-fab1442dc7a2 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:56.867033 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cdc15ba3-861b-4ede-b963-fab1442dc7a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:56.922986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc15ba3-861b-4ede-b963-fab1442dc7a2 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:56.926616 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e8a6f-81cd-4a49-b7eb-812ad63bce7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:56.927801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e8a6f-81cd-4a49-b7eb-812ad63bce7b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:56.927801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e8a6f-81cd-4a49-b7eb-812ad63bce7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:56.927987 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e8a6f-81cd-4a49-b7eb-812ad63bce7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:56.929780 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ef1e8a6f-81cd-4a49-b7eb-812ad63bce7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:57.002241 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e8a6f-81cd-4a49-b7eb-812ad63bce7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:57.003807 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e8a6f-81cd-4a49-b7eb-812ad63bce7b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:57.016605 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1409/2835] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:58.055840 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:58.211307 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa077d-a78f-495f-b84f-50259c71f9fe None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:58.212735 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-beaa077d-a78f-495f-b84f-50259c71f9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:58.225263 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa077d-a78f-495f-b84f-50259c71f9fe None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:58.228799 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1387e-9b8a-4d66-94f7-5d0514d18ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:58.229436 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1387e-9b8a-4d66-94f7-5d0514d18ecd service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.229719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1387e-9b8a-4d66-94f7-5d0514d18ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.229996 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1387e-9b8a-4d66-94f7-5d0514d18ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.231728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5bd1387e-9b8a-4d66-94f7-5d0514d18ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:58.244610 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1387e-9b8a-4d66-94f7-5d0514d18ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:58.246024 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1387e-9b8a-4d66-94f7-5d0514d18ecd service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:58.256747 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1427/2836] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:03.242504 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8948d4f-a825-42ee-8972-91ae36db7ec5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:03.242983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8948d4f-a825-42ee-8972-91ae36db7ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:03.243083 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8948d4f-a825-42ee-8972-91ae36db7ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:03.261397 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a8948d4f-a825-42ee-8972-91ae36db7ec5 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:03.262622 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a8948d4f-a825-42ee-8972-91ae36db7ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:03.316211 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1410/2837] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:12:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:12:03.361246 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1387e-9b8a-4d66-94f7-5d0514d18ecd service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:03.362733 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5bd1387e-9b8a-4d66-94f7-5d0514d18ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:03.431572 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1387e-9b8a-4d66-94f7-5d0514d18ecd service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:03.436566 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010beb9-67b5-47a9-9438-0865a9cf9391 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:03.437209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010beb9-67b5-47a9-9438-0865a9cf9391 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:03.437494 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010beb9-67b5-47a9-9438-0865a9cf9391 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:03.437776 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010beb9-67b5-47a9-9438-0865a9cf9391 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:03.446485 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010beb9-67b5-47a9-9438-0865a9cf9391 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:03.448141 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010beb9-67b5-47a9-9438-0865a9cf9391 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:03.456106 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1428/2838] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:12:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:12:03.948758 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc4848-6dd8-4299-a6c0-3c6132aa25b9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:03.949053 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc4848-6dd8-4299-a6c0-3c6132aa25b9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:03.949367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc4848-6dd8-4299-a6c0-3c6132aa25b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:03.967362 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-64cc4848-6dd8-4299-a6c0-3c6132aa25b9 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:03.968582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-64cc4848-6dd8-4299-a6c0-3c6132aa25b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:03.976314 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1411/2839] 10.222.0.65 () {58 vars in 967 bytes} [Thu Jun 23 11:12:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:12:03.988112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010beb9-67b5-47a9-9438-0865a9cf9391 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:03.998774 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010beb9-67b5-47a9-9438-0865a9cf9391 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:04.004158 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eff9f0-392a-4db6-aada-6ba46ead96f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:04.004812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eff9f0-392a-4db6-aada-6ba46ead96f7 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:04.005091 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eff9f0-392a-4db6-aada-6ba46ead96f7 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:04.005364 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eff9f0-392a-4db6-aada-6ba46ead96f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:04.013457 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eff9f0-392a-4db6-aada-6ba46ead96f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:04.015018 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eff9f0-392a-4db6-aada-6ba46ead96f7 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:04.023797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1429/2840] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:12:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:12:04.206736 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7169358-bc00-4795-ade7-4bf2084dc3be None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:04.207022 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7169358-bc00-4795-ade7-4bf2084dc3be None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:04.207301 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7169358-bc00-4795-ade7-4bf2084dc3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:04.225912 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f7169358-bc00-4795-ade7-4bf2084dc3be None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:04.227111 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f7169358-bc00-4795-ade7-4bf2084dc3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:04.234705 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1412/2841] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:12:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:04.246257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eff9f0-392a-4db6-aada-6ba46ead96f7 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:04.256811 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eff9f0-392a-4db6-aada-6ba46ead96f7 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:04.261791 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78128bd-13e0-4849-b91e-189732407093 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:04.262420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78128bd-13e0-4849-b91e-189732407093 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:04.262825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78128bd-13e0-4849-b91e-189732407093 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:04.263169 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78128bd-13e0-4849-b91e-189732407093 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:04.271670 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78128bd-13e0-4849-b91e-189732407093 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:04.273502 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78128bd-13e0-4849-b91e-189732407093 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:04.281434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1430/2842] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:12:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:12:04.590034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac658c8-6caf-494b-9c4a-bea950b555ee None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:04.590494 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac658c8-6caf-494b-9c4a-bea950b555ee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:04.590597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac658c8-6caf-494b-9c4a-bea950b555ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:04.608763 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3ac658c8-6caf-494b-9c4a-bea950b555ee None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:04.609946 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3ac658c8-6caf-494b-9c4a-bea950b555ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:04.617076 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1413/2843] 10.222.0.65 () {58 vars in 967 bytes} [Thu Jun 23 11:12:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:04.628678 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78128bd-13e0-4849-b91e-189732407093 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:04.639688 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78128bd-13e0-4849-b91e-189732407093 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:04.644659 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e813f3d-eb98-44d8-9596-0e2b5eadb735 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:04.645928 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e813f3d-eb98-44d8-9596-0e2b5eadb735 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:04.646209 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e813f3d-eb98-44d8-9596-0e2b5eadb735 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:04.646487 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e813f3d-eb98-44d8-9596-0e2b5eadb735 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:04.654668 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e813f3d-eb98-44d8-9596-0e2b5eadb735 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:04.656276 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e813f3d-eb98-44d8-9596-0e2b5eadb735 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:04.664288 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1431/2844] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:12:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:12:06.681252 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac658c8-6caf-494b-9c4a-bea950b555ee None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:06.709950 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:06.744680 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac658c8-6caf-494b-9c4a-bea950b555ee None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:06.749327 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:06.750199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:06.750501 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:06.750779 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] PATH_INFO: `/v3/users/851f30b7cc36417986029dfcc40b124f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:06.752001 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] RBAC: Authorizing `identity:delete_user(user_id=851f30b7cc36417986029dfcc40b124f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:06.753928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:06.798723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '851f30b7cc36417986029dfcc40b124f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:06.799467 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '851f30b7cc36417986029dfcc40b124f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:06.807986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '851f30b7cc36417986029dfcc40b124f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:06.819962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '851f30b7cc36417986029dfcc40b124f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:06.826635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '851f30b7cc36417986029dfcc40b124f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:06.832462 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1414/2845] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:06 2022] DELETE /identity/v3/users/851f30b7cc36417986029dfcc40b124f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:06.837155 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea3099-88a2-4c61-8bdf-c41644257727 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:06.837623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea3099-88a2-4c61-8bdf-c41644257727 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:06.837746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea3099-88a2-4c61-8bdf-c41644257727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:06.855006 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cfea3099-88a2-4c61-8bdf-c41644257727 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:06.856222 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cfea3099-88a2-4c61-8bdf-c41644257727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:06.912825 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1432/2846] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:06.922280 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:06.923693 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:06.980803 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe706e3c-ce13-47c6-a134-7b0cebe5f74b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:06.984798 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de722340-844c-4c84-bdca-476c5f00692a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:06.985661 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de722340-844c-4c84-bdca-476c5f00692a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:06.986136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de722340-844c-4c84-bdca-476c5f00692a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:06.986500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de722340-844c-4c84-bdca-476c5f00692a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:06.996110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de722340-844c-4c84-bdca-476c5f00692a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:06.997532 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de722340-844c-4c84-bdca-476c5f00692a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:07.006894 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1415/2847] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:07.175511 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea3099-88a2-4c61-8bdf-c41644257727 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:07.177054 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cfea3099-88a2-4c61-8bdf-c41644257727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:07.188967 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea3099-88a2-4c61-8bdf-c41644257727 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:07.193529 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24edf4-03de-4d70-b66c-a9942d823509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:07.194394 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24edf4-03de-4d70-b66c-a9942d823509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.194686 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24edf4-03de-4d70-b66c-a9942d823509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.194951 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24edf4-03de-4d70-b66c-a9942d823509 admin admin] PATH_INFO: `/v3/projects/f5c3243de4c94fe99d6dd7f96c814e62` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.196182 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24edf4-03de-4d70-b66c-a9942d823509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5c3243de4c94fe99d6dd7f96c814e62)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:07.198089 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24edf4-03de-4d70-b66c-a9942d823509 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:07.233935 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:07.247053 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9d24edf4-03de-4d70-b66c-a9942d823509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5c3243de4c94fe99d6dd7f96c814e62', 'request_id': 'req-9d24edf4-03de-4d70-b66c-a9942d823509'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:07.253548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9d24edf4-03de-4d70-b66c-a9942d823509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5c3243de4c94fe99d6dd7f96c814e62', 'request_id': 'req-9d24edf4-03de-4d70-b66c-a9942d823509'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:07.260008 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9d24edf4-03de-4d70-b66c-a9942d823509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5c3243de4c94fe99d6dd7f96c814e62', 'request_id': 'req-9d24edf4-03de-4d70-b66c-a9942d823509'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:07.273081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9d24edf4-03de-4d70-b66c-a9942d823509 admin admin] The token cache is being invalidate because project f5c3243de4c94fe99d6dd7f96c814e62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:07.273371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9d24edf4-03de-4d70-b66c-a9942d823509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:07.273995 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9d24edf4-03de-4d70-b66c-a9942d823509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:07.275539 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1433/2848] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:07 2022] DELETE /identity/v3/projects/f5c3243de4c94fe99d6dd7f96c814e62 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:07.280631 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbfecb-3366-4c83-aacd-8eb939940442 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.280925 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbfecb-3366-4c83-aacd-8eb939940442 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.281205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbfecb-3366-4c83-aacd-8eb939940442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.299546 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f6fbfecb-3366-4c83-aacd-8eb939940442 None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:07.300787 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f6fbfecb-3366-4c83-aacd-8eb939940442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:07.345206 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:07.361535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1416/2849] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:07.369408 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67c680-714c-463e-ac3a-7c91a221dc8c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.369702 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67c680-714c-463e-ac3a-7c91a221dc8c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.369983 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67c680-714c-463e-ac3a-7c91a221dc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.388330 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4f67c680-714c-463e-ac3a-7c91a221dc8c None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:07.389531 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4f67c680-714c-463e-ac3a-7c91a221dc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:07.398166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1434/2850] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:07.402547 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbfecb-3366-4c83-aacd-8eb939940442 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:07.412152 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbfecb-3366-4c83-aacd-8eb939940442 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:07.416708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf0005-b5be-4a6e-9026-99f7cc9da7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:07.417352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf0005-b5be-4a6e-9026-99f7cc9da7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.417684 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf0005-b5be-4a6e-9026-99f7cc9da7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.417907 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf0005-b5be-4a6e-9026-99f7cc9da7e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.418621 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf0005-b5be-4a6e-9026-99f7cc9da7e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:12:07.418928 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf0005-b5be-4a6e-9026-99f7cc9da7e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:07.420558 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf0005-b5be-4a6e-9026-99f7cc9da7e5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:07.431778 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1417/2851] 10.222.0.65 () {58 vars in 1152 bytes} [Thu Jun 23 11:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:12:07.435916 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67c680-714c-463e-ac3a-7c91a221dc8c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:07.446306 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67c680-714c-463e-ac3a-7c91a221dc8c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:07.451036 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c4189-f0e1-442b-8afa-309fc0c941e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:07.451710 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c4189-f0e1-442b-8afa-309fc0c941e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.452001 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c4189-f0e1-442b-8afa-309fc0c941e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.452281 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c4189-f0e1-442b-8afa-309fc0c941e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.452972 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c4189-f0e1-442b-8afa-309fc0c941e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:07.454855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c4189-f0e1-442b-8afa-309fc0c941e3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:07.480314 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1435/2852] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:12:07 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:07.484373 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf0005-b5be-4a6e-9026-99f7cc9da7e5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:07.494282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf0005-b5be-4a6e-9026-99f7cc9da7e5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:07.549942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852b4b7-244a-4959-aa65-df5439c42d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:07.550609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852b4b7-244a-4959-aa65-df5439c42d85 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.550896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852b4b7-244a-4959-aa65-df5439c42d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.551178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852b4b7-244a-4959-aa65-df5439c42d85 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.551902 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852b4b7-244a-4959-aa65-df5439c42d85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:07.553805 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852b4b7-244a-4959-aa65-df5439c42d85 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:07.574324 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1418/2853] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:12:07 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:07.578874 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c4189-f0e1-442b-8afa-309fc0c941e3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:07.589853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c4189-f0e1-442b-8afa-309fc0c941e3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:07.594437 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35e15f-7bb3-42fb-9578-a158e65b0347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:07.595100 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35e15f-7bb3-42fb-9578-a158e65b0347 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.595386 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35e15f-7bb3-42fb-9578-a158e65b0347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.595682 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35e15f-7bb3-42fb-9578-a158e65b0347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.596428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35e15f-7bb3-42fb-9578-a158e65b0347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:07.598016 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35e15f-7bb3-42fb-9578-a158e65b0347 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:07.608253 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1436/2854] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:07.611951 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852b4b7-244a-4959-aa65-df5439c42d85 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:07.622016 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852b4b7-244a-4959-aa65-df5439c42d85 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:07.626613 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db257f-83a4-466f-9499-5e9cca063351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:07.627262 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db257f-83a4-466f-9499-5e9cca063351 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.627549 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db257f-83a4-466f-9499-5e9cca063351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.627865 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db257f-83a4-466f-9499-5e9cca063351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.628602 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db257f-83a4-466f-9499-5e9cca063351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:07.630676 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db257f-83a4-466f-9499-5e9cca063351 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:07.641609 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1419/2855] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:07.647537 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35e15f-7bb3-42fb-9578-a158e65b0347 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:07.659709 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35e15f-7bb3-42fb-9578-a158e65b0347 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:07.664410 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00a903-24db-414f-bdd8-54f378854fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:07.665295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00a903-24db-414f-bdd8-54f378854fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.665582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00a903-24db-414f-bdd8-54f378854fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.665866 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00a903-24db-414f-bdd8-54f378854fb7 admin admin] PATH_INFO: `/v3/projects/0d0d343d1ec648a981cc8d7910b92401/users/7d3495251b56423fbe694d5cb7ef4695/roles/7d15e336f8034a1fbdef48cfb7c4f717` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.680746 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00a903-24db-414f-bdd8-54f378854fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0d343d1ec648a981cc8d7910b92401, user_id=7d3495251b56423fbe694d5cb7ef4695, role_id=7d15e336f8034a1fbdef48cfb7c4f717)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:07.683323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00a903-24db-414f-bdd8-54f378854fb7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:07.696812 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1437/2856] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:07 2022] PUT /identity/v3/projects/0d0d343d1ec648a981cc8d7910b92401/users/7d3495251b56423fbe694d5cb7ef4695/roles/7d15e336f8034a1fbdef48cfb7c4f717 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:12:07.701791 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cadd8-0bbc-426c-a018-4696cda6cdca None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.702258 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cadd8-0bbc-426c-a018-4696cda6cdca None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.702380 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cadd8-0bbc-426c-a018-4696cda6cdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.720297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-200cadd8-0bbc-426c-a018-4696cda6cdca None None] MFA Rules not processed for user `7c47db9f01c84c63b57cf3ae5711ffb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:07.721681 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-200cadd8-0bbc-426c-a018-4696cda6cdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:07.781679 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1420/2857] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:12:07.791108 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00a903-24db-414f-bdd8-54f378854fb7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:07.792877 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0a00a903-24db-414f-bdd8-54f378854fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:07.849548 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00a903-24db-414f-bdd8-54f378854fb7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:07.853201 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfea658-290f-4f37-a28e-9e19b54a89a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:07.853850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfea658-290f-4f37-a28e-9e19b54a89a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.854136 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfea658-290f-4f37-a28e-9e19b54a89a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.854418 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfea658-290f-4f37-a28e-9e19b54a89a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.863349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfea658-290f-4f37-a28e-9e19b54a89a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:07.865324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfea658-290f-4f37-a28e-9e19b54a89a1 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:07.875415 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1438/2858] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:11.569766 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d697d-5326-4e38-a6b0-a7357b0c05cb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:11.570316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d697d-5326-4e38-a6b0-a7357b0c05cb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:11.570316 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d697d-5326-4e38-a6b0-a7357b0c05cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:11.599574 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5e3d697d-5326-4e38-a6b0-a7357b0c05cb None None] MFA Rules not processed for user `7d3495251b56423fbe694d5cb7ef4695`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:11.600926 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5e3d697d-5326-4e38-a6b0-a7357b0c05cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:11.648518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:11.672695 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1421/2859] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:12:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:11.677302 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfea658-290f-4f37-a28e-9e19b54a89a1 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:11.687365 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfea658-290f-4f37-a28e-9e19b54a89a1 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:11.692404 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa953c9-e797-4c45-9352-661b36f3c557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:11.693184 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa953c9-e797-4c45-9352-661b36f3c557 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:11.693530 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa953c9-e797-4c45-9352-661b36f3c557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:11.693889 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa953c9-e797-4c45-9352-661b36f3c557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:11.694738 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa953c9-e797-4c45-9352-661b36f3c557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:11.696823 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa953c9-e797-4c45-9352-661b36f3c557 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:11.725087 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1439/2860] 10.222.0.65 () {60 vars in 1152 bytes} [Thu Jun 23 11:12:11 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:11.729207 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d697d-5326-4e38-a6b0-a7357b0c05cb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:11.740641 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d697d-5326-4e38-a6b0-a7357b0c05cb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:11.802025 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daf650-f6f4-48f1-89ba-8f60906b7f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:11.802851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daf650-f6f4-48f1-89ba-8f60906b7f33 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:11.803140 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daf650-f6f4-48f1-89ba-8f60906b7f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:11.803418 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daf650-f6f4-48f1-89ba-8f60906b7f33 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:11.804172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47daf650-f6f4-48f1-89ba-8f60906b7f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:11.806113 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47daf650-f6f4-48f1-89ba-8f60906b7f33 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:11.829857 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1422/2861] 10.222.0.65 () {60 vars in 1143 bytes} [Thu Jun 23 11:12:11 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:11.834705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa953c9-e797-4c45-9352-661b36f3c557 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:11.845546 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa953c9-e797-4c45-9352-661b36f3c557 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:11.850323 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67256eb-fdb7-494f-83e4-4babce535453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:11.850997 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67256eb-fdb7-494f-83e4-4babce535453 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:11.851281 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67256eb-fdb7-494f-83e4-4babce535453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:11.851563 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67256eb-fdb7-494f-83e4-4babce535453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:11.852339 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67256eb-fdb7-494f-83e4-4babce535453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:11.854301 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67256eb-fdb7-494f-83e4-4babce535453 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:11.868121 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1440/2862] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:11.872149 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daf650-f6f4-48f1-89ba-8f60906b7f33 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:11.883230 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daf650-f6f4-48f1-89ba-8f60906b7f33 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:11.888036 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fafec6-5fb2-4c18-885f-97d412a8664f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:11.888939 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fafec6-5fb2-4c18-885f-97d412a8664f admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:11.889220 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fafec6-5fb2-4c18-885f-97d412a8664f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:11.889500 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fafec6-5fb2-4c18-885f-97d412a8664f admin admin] PATH_INFO: `/v3/projects/5f0e61e86cbc44508ae1f7286a1f1713/users/624c46e0e0864a288888ba022db8b84f/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:11.908192 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fafec6-5fb2-4c18-885f-97d412a8664f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0e61e86cbc44508ae1f7286a1f1713, user_id=624c46e0e0864a288888ba022db8b84f, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:11.910961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fafec6-5fb2-4c18-885f-97d412a8664f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:11.925116 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1423/2863] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:11 2022] PUT /identity/v3/projects/5f0e61e86cbc44508ae1f7286a1f1713/users/624c46e0e0864a288888ba022db8b84f/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:11.929112 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67256eb-fdb7-494f-83e4-4babce535453 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:11.939291 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67256eb-fdb7-494f-83e4-4babce535453 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:11.964645 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:12.001259 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c62dc-fafa-49db-9d2f-8a6412c80653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:12.001858 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c62dc-fafa-49db-9d2f-8a6412c80653 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:12.002153 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c62dc-fafa-49db-9d2f-8a6412c80653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:12.002434 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c62dc-fafa-49db-9d2f-8a6412c80653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:12.003176 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c62dc-fafa-49db-9d2f-8a6412c80653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:12.005385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c62dc-fafa-49db-9d2f-8a6412c80653 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:12.015998 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1441/2864] 10.222.0.65 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:12.020795 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fafec6-5fb2-4c18-885f-97d412a8664f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:12.031367 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fafec6-5fb2-4c18-885f-97d412a8664f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:12.035974 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14020b2-74ca-47ab-a933-a9d886a9e585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:12.036852 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14020b2-74ca-47ab-a933-a9d886a9e585 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:12.037134 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14020b2-74ca-47ab-a933-a9d886a9e585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:12.037410 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14020b2-74ca-47ab-a933-a9d886a9e585 admin admin] PATH_INFO: `/v3/projects/5f0e61e86cbc44508ae1f7286a1f1713/users/624c46e0e0864a288888ba022db8b84f/roles/266f01c0faa946f0acfca0aac74f2461` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:12.039472 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14020b2-74ca-47ab-a933-a9d886a9e585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0e61e86cbc44508ae1f7286a1f1713, user_id=624c46e0e0864a288888ba022db8b84f, role_id=266f01c0faa946f0acfca0aac74f2461)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:12.042136 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14020b2-74ca-47ab-a933-a9d886a9e585 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:12.052034 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1424/2865] 10.222.0.65 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:12 2022] PUT /identity/v3/projects/5f0e61e86cbc44508ae1f7286a1f1713/users/624c46e0e0864a288888ba022db8b84f/roles/266f01c0faa946f0acfca0aac74f2461 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:12:16.996560 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353269c8-f732-446e-af64-2361ac79bf5b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:16.997105 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353269c8-f732-446e-af64-2361ac79bf5b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:16.997366 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353269c8-f732-446e-af64-2361ac79bf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:17.037382 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-353269c8-f732-446e-af64-2361ac79bf5b None None] MFA Rules not processed for user `624c46e0e0864a288888ba022db8b84f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:17.038647 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-353269c8-f732-446e-af64-2361ac79bf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:17.127803 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1442/2866] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:12:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:17.138052 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14020b2-74ca-47ab-a933-a9d886a9e585 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:17.139799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d14020b2-74ca-47ab-a933-a9d886a9e585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:17.200582 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14020b2-74ca-47ab-a933-a9d886a9e585 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:17.204243 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117ffc6-9c4b-4274-917a-cd10b85f2952 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5af6cc8c56db48beae2550954446c119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:17.204936 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117ffc6-9c4b-4274-917a-cd10b85f2952 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:17.205225 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117ffc6-9c4b-4274-917a-cd10b85f2952 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:17.205506 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117ffc6-9c4b-4274-917a-cd10b85f2952 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:17.256573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:17.267720 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117ffc6-9c4b-4274-917a-cd10b85f2952 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:17.269176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117ffc6-9c4b-4274-917a-cd10b85f2952 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:17.282946 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1425/2867] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:17.382154 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a22c41-32f3-4c27-8617-67f233dcddda None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:17.382297 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a22c41-32f3-4c27-8617-67f233dcddda None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:17.382579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a22c41-32f3-4c27-8617-67f233dcddda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:17.400824 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e8a22c41-32f3-4c27-8617-67f233dcddda None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:17.402085 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e8a22c41-32f3-4c27-8617-67f233dcddda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:17.462348 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1443/2868] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:12:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:12:17.474851 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117ffc6-9c4b-4274-917a-cd10b85f2952 service glance] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:17.476426 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f117ffc6-9c4b-4274-917a-cd10b85f2952 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:17.549489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117ffc6-9c4b-4274-917a-cd10b85f2952 service glance] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:17.554603 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdccfce-a135-4d9a-a0de-8376605be8ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:17.555249 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdccfce-a135-4d9a-a0de-8376605be8ed service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:17.555535 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdccfce-a135-4d9a-a0de-8376605be8ed service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:17.555844 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdccfce-a135-4d9a-a0de-8376605be8ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:17.563801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdccfce-a135-4d9a-a0de-8376605be8ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:17.565437 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdccfce-a135-4d9a-a0de-8376605be8ed service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:17.573523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1426/2869] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:12:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:12:26.930894 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a40819-ede2-4d60-9a60-d9b4aadca2c6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:26.931399 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a40819-ede2-4d60-9a60-d9b4aadca2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:26.931501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a40819-ede2-4d60-9a60-d9b4aadca2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:26.950022 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-61a40819-ede2-4d60-9a60-d9b4aadca2c6 None None] MFA Rules not processed for user `7d3495251b56423fbe694d5cb7ef4695`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:26.951385 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-61a40819-ede2-4d60-9a60-d9b4aadca2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:26.958719 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1444/2870] 10.222.0.65 () {58 vars in 957 bytes} [Thu Jun 23 11:12:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:27.570231 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=36 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:30.021653 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d03049-6d79-4a6a-934b-3f3892be7687 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:30.022283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d03049-6d79-4a6a-934b-3f3892be7687 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:30.022283 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d03049-6d79-4a6a-934b-3f3892be7687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:30.041148 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d2d03049-6d79-4a6a-934b-3f3892be7687 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:30.042440 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d2d03049-6d79-4a6a-934b-3f3892be7687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:30.050761 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1427/2871] 10.222.0.65 () {58 vars in 1009 bytes} [Thu Jun 23 11:12:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:30.064691 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a40819-ede2-4d60-9a60-d9b4aadca2c6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:30.078057 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a40819-ede2-4d60-9a60-d9b4aadca2c6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:30.084247 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a6619-cf79-42ba-9af1-cd6d277ef487 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:30.085023 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a6619-cf79-42ba-9af1-cd6d277ef487 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:30.085384 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a6619-cf79-42ba-9af1-cd6d277ef487 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:30.085840 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a6619-cf79-42ba-9af1-cd6d277ef487 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:30.106293 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a6619-cf79-42ba-9af1-cd6d277ef487 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:30.108162 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a6619-cf79-42ba-9af1-cd6d277ef487 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:30.121164 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1445/2872] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:12:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 11:12:40.035193 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:40.115558 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:41.309942 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d03049-6d79-4a6a-934b-3f3892be7687 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:41.311485 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d2d03049-6d79-4a6a-934b-3f3892be7687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:41.367934 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d03049-6d79-4a6a-934b-3f3892be7687 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:41.371394 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8b30b-9faa-4d92-a3a2-6ad4e57332c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:41.372078 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8b30b-9faa-4d92-a3a2-6ad4e57332c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:41.372362 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8b30b-9faa-4d92-a3a2-6ad4e57332c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:41.372635 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8b30b-9faa-4d92-a3a2-6ad4e57332c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:41.374387 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0fc8b30b-9faa-4d92-a3a2-6ad4e57332c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:41.436199 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8b30b-9faa-4d92-a3a2-6ad4e57332c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:41.437629 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8b30b-9faa-4d92-a3a2-6ad4e57332c1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:41.447368 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1428/2873] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:47.523336 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a6619-cf79-42ba-9af1-cd6d277ef487 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:47.525734 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-726a6619-cf79-42ba-9af1-cd6d277ef487 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:47.541263 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a6619-cf79-42ba-9af1-cd6d277ef487 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:47.545815 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:47.546687 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:47.546971 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:47.547249 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] PATH_INFO: `/v3/users/8f8da2142b9e4751b436cfd4920587a0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:47.548458 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f8da2142b9e4751b436cfd4920587a0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:47.550379 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:47.598743 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f8da2142b9e4751b436cfd4920587a0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:47.599430 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f8da2142b9e4751b436cfd4920587a0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:47.600081 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f8da2142b9e4751b436cfd4920587a0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:47.612321 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f8da2142b9e4751b436cfd4920587a0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:47.620216 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f8da2142b9e4751b436cfd4920587a0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:47.637217 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1446/2874] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:47 2022] DELETE /identity/v3/users/8f8da2142b9e4751b436cfd4920587a0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:47.839540 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8b30b-9faa-4d92-a3a2-6ad4e57332c1 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:47.850121 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8b30b-9faa-4d92-a3a2-6ad4e57332c1 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:47.914745 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:47.915615 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:47.916004 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:47.916209 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] PATH_INFO: `/v3/projects/1fccc592269f433b99c2d13dad5e8e39` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:47.917404 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fccc592269f433b99c2d13dad5e8e39)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:47.919415 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:47.972799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fccc592269f433b99c2d13dad5e8e39', 'request_id': 'req-5ce0042c-9883-4beb-9d4f-a2ea540361da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:47.976352 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:47.984146 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fccc592269f433b99c2d13dad5e8e39', 'request_id': 'req-5ce0042c-9883-4beb-9d4f-a2ea540361da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:47.992110 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fccc592269f433b99c2d13dad5e8e39', 'request_id': 'req-5ce0042c-9883-4beb-9d4f-a2ea540361da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.003350 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] The token cache is being invalidate because project 1fccc592269f433b99c2d13dad5e8e39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:48.003639 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.004322 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.005977 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1429/2875] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:47 2022] DELETE /identity/v3/projects/1fccc592269f433b99c2d13dad5e8e39 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:48.009980 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:48.011377 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:48.025257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844a625-cbbc-4ce4-8390-d5bb80df4057 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:48.082495 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:48.083407 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:48.083723 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:48.083998 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] PATH_INFO: `/v3/users/4e41807dbd8a4922ba199f4bab63a9b5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:48.085198 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e41807dbd8a4922ba199f4bab63a9b5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:48.087166 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:48.141535 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e41807dbd8a4922ba199f4bab63a9b5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.143774 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e41807dbd8a4922ba199f4bab63a9b5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.143774 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e41807dbd8a4922ba199f4bab63a9b5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.158616 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e41807dbd8a4922ba199f4bab63a9b5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.168037 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e41807dbd8a4922ba199f4bab63a9b5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.170095 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:48.186643 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1447/2876] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:48 2022] DELETE /identity/v3/users/4e41807dbd8a4922ba199f4bab63a9b5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:12:48.414595 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:48.416130 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:48.436242 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0042c-9883-4beb-9d4f-a2ea540361da admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:48.505589 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:48.507883 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:48.508183 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:48.508493 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] PATH_INFO: `/v3/projects/80e78c5ecac645ffbc1a9dbc3aa0f82e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:48.509770 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e78c5ecac645ffbc1a9dbc3aa0f82e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:48.513054 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:48.576717 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e78c5ecac645ffbc1a9dbc3aa0f82e', 'request_id': 'req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.586282 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e78c5ecac645ffbc1a9dbc3aa0f82e', 'request_id': 'req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.593384 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e78c5ecac645ffbc1a9dbc3aa0f82e', 'request_id': 'req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.608107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] The token cache is being invalidate because project 80e78c5ecac645ffbc1a9dbc3aa0f82e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:48.608489 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.609171 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:48.610754 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1430/2877] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:48 2022] DELETE /identity/v3/projects/80e78c5ecac645ffbc1a9dbc3aa0f82e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:57.862584 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:57.864090 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:57.920595 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa3653-66c5-4f32-a63f-ce1bf2e4c12c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:57.967371 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84565a-d201-4ff7-915b-f32752cad20b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:57.968045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84565a-d201-4ff7-915b-f32752cad20b service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:57.968335 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84565a-d201-4ff7-915b-f32752cad20b service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:57.968615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84565a-d201-4ff7-915b-f32752cad20b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:57.970355 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5b84565a-d201-4ff7-915b-f32752cad20b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:58.031855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84565a-d201-4ff7-915b-f32752cad20b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:58.033234 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84565a-d201-4ff7-915b-f32752cad20b service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:58.043319 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1448/2878] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:58.604723 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:59.787518 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:59.789089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:59.877178 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ca4aa-9d99-476e-a1f1-e3b08853fc2b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:59.883802 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5a09f-e9e3-468e-ae96-a3c08b4265e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:59.884774 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5a09f-e9e3-468e-ae96-a3c08b4265e4 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:59.885299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5a09f-e9e3-468e-ae96-a3c08b4265e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:59.885785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5a09f-e9e3-468e-ae96-a3c08b4265e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:59.887917 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-44e5a09f-e9e3-468e-ae96-a3c08b4265e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:59.902644 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5a09f-e9e3-468e-ae96-a3c08b4265e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:59.904515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5a09f-e9e3-468e-ae96-a3c08b4265e4 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:59.917682 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1431/2879] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:13:00.016206 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84565a-d201-4ff7-915b-f32752cad20b service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:13:00.017703 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5b84565a-d201-4ff7-915b-f32752cad20b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:13:00.031025 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84565a-d201-4ff7-915b-f32752cad20b service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:13:00.034868 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2608e33-dc39-47de-8fce-480712f3930c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:13:00.035724 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2608e33-dc39-47de-8fce-480712f3930c service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:13:00.036015 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2608e33-dc39-47de-8fce-480712f3930c service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:13:00.036990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2608e33-dc39-47de-8fce-480712f3930c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:13:00.038413 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f2608e33-dc39-47de-8fce-480712f3930c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:13:00.058256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2608e33-dc39-47de-8fce-480712f3930c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:13:00.060211 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2608e33-dc39-47de-8fce-480712f3930c service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:13:00.071175 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1449/2880] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:13:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:13:09.913667 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=34 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:13:10.066568 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=78 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:13:56.143479 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7255820-79ad-43b8-925e-53131e1f0fed None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:13:56.144114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7255820-79ad-43b8-925e-53131e1f0fed None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:13:56.144114 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7255820-79ad-43b8-925e-53131e1f0fed None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:13:56.145597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1432/2881] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 11:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:13:56.152613 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232ac6-e8e7-46a6-9e20-274ad75a4cc0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:13:56.153358 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232ac6-e8e7-46a6-9e20-274ad75a4cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:13:56.157021 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232ac6-e8e7-46a6-9e20-274ad75a4cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:13:56.194423 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-45232ac6-e8e7-46a6-9e20-274ad75a4cc0 None None] MFA Rules not processed for user `e96b136657864104a51900debd1fbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:13:56.195646 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-45232ac6-e8e7-46a6-9e20-274ad75a4cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:13:56.202860 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1450/2882] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 11:13:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:13:56.215514 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7255820-79ad-43b8-925e-53131e1f0fed None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:13:56.217304 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e7255820-79ad-43b8-925e-53131e1f0fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:13:56.230620 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7255820-79ad-43b8-925e-53131e1f0fed None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:13:56.234642 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc4aef-3879-4d9e-9726-30b544bed68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:13:56.235374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc4aef-3879-4d9e-9726-30b544bed68c service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:13:56.235757 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc4aef-3879-4d9e-9726-30b544bed68c service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:13:56.236107 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc4aef-3879-4d9e-9726-30b544bed68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:13:56.243510 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc4aef-3879-4d9e-9726-30b544bed68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:13:56.245067 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc4aef-3879-4d9e-9726-30b544bed68c service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:13:56.253172 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1433/2883] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:13:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:13:56.334622 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90c11b-030e-48e4-ada4-e74cc90e5088 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:13:56.334912 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90c11b-030e-48e4-ada4-e74cc90e5088 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:13:56.335192 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90c11b-030e-48e4-ada4-e74cc90e5088 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:13:56.336597 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1451/2884] 10.222.0.65 () {54 vars in 921 bytes} [Thu Jun 23 11:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:13:56.342831 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47bebd-c061-42d8-bf0c-9c86ceaa97e0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:13:56.343120 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47bebd-c061-42d8-bf0c-9c86ceaa97e0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:13:56.343400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47bebd-c061-42d8-bf0c-9c86ceaa97e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:13:56.362455 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3e47bebd-c061-42d8-bf0c-9c86ceaa97e0 None None] MFA Rules not processed for user `b01d875111bd43698bcf99d30f7a7f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:13:56.363586 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3e47bebd-c061-42d8-bf0c-9c86ceaa97e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:13:56.373285 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1434/2885] 10.222.0.65 () {58 vars in 1018 bytes} [Thu Jun 23 11:13:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:13:56.383501 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90c11b-030e-48e4-ada4-e74cc90e5088 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:13:56.394513 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90c11b-030e-48e4-ada4-e74cc90e5088 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:13:56.398118 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e444ff-78c7-42ee-b0f0-a9a09a82aa9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e96b136657864104a51900debd1fbe04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:13:56.398798 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e444ff-78c7-42ee-b0f0-a9a09a82aa9f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:13:56.399110 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e444ff-78c7-42ee-b0f0-a9a09a82aa9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:13:56.399388 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e444ff-78c7-42ee-b0f0-a9a09a82aa9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:13:56.410254 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e444ff-78c7-42ee-b0f0-a9a09a82aa9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:13:56.411728 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e444ff-78c7-42ee-b0f0-a9a09a82aa9f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:13:56.423103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1452/2886] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:13:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:13:56.483163 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47bebd-c061-42d8-bf0c-9c86ceaa97e0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:13:56.484585 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3e47bebd-c061-42d8-bf0c-9c86ceaa97e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:13:56.551646 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47bebd-c061-42d8-bf0c-9c86ceaa97e0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:13:56.556713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04285342-676d-47ac-b09d-f98681818f91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:13:56.557350 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04285342-676d-47ac-b09d-f98681818f91 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:13:56.557633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04285342-676d-47ac-b09d-f98681818f91 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:13:56.557911 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04285342-676d-47ac-b09d-f98681818f91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:13:56.559713 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-04285342-676d-47ac-b09d-f98681818f91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:13:56.570935 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04285342-676d-47ac-b09d-f98681818f91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:13:56.572708 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04285342-676d-47ac-b09d-f98681818f91 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:13:56.582515 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1435/2887] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:13:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:13:57.613599 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e444ff-78c7-42ee-b0f0-a9a09a82aa9f service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:13:57.625084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e444ff-78c7-42ee-b0f0-a9a09a82aa9f service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:13:57.630618 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7b0f0-bf11-48ce-b61e-0fd484aa7410 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:13:57.631298 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7b0f0-bf11-48ce-b61e-0fd484aa7410 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:13:57.631582 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7b0f0-bf11-48ce-b61e-0fd484aa7410 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:13:57.631855 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7b0f0-bf11-48ce-b61e-0fd484aa7410 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:13:57.633705 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fbe7b0f0-bf11-48ce-b61e-0fd484aa7410 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:13:57.646848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7b0f0-bf11-48ce-b61e-0fd484aa7410 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:13:57.648471 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7b0f0-bf11-48ce-b61e-0fd484aa7410 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:13:57.660729 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1453/2888] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:13:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:14:06.578235 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=41 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:14:07.655764 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=21 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:14:15.462297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04285342-676d-47ac-b09d-f98681818f91 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:14:15.463896 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-04285342-676d-47ac-b09d-f98681818f91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:14:15.486884 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04285342-676d-47ac-b09d-f98681818f91 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:14:15.502438 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06082bd6-aa20-4a45-bc60-c6e89533616e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01d875111bd43698bcf99d30f7a7f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:14:15.503181 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06082bd6-aa20-4a45-bc60-c6e89533616e service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:14:15.503570 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06082bd6-aa20-4a45-bc60-c6e89533616e service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:14:15.503892 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06082bd6-aa20-4a45-bc60-c6e89533616e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:14:15.509923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-06082bd6-aa20-4a45-bc60-c6e89533616e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:14:15.524767 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06082bd6-aa20-4a45-bc60-c6e89533616e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:14:15.526856 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06082bd6-aa20-4a45-bc60-c6e89533616e service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:14:15.543787 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1436/2889] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:14:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:14:23.246934 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f65ac-f8dd-40fd-ace3-037019e6e407 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:14:23.247418 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f65ac-f8dd-40fd-ace3-037019e6e407 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:14:23.247526 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f65ac-f8dd-40fd-ace3-037019e6e407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:14:23.265048 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c74f65ac-f8dd-40fd-ace3-037019e6e407 None None] MFA Rules not processed for user `96655b920bda45cb872624aa4183cb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:14:23.266271 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c74f65ac-f8dd-40fd-ace3-037019e6e407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:14:23.321188 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1454/2890] 10.222.0.65 () {58 vars in 967 bytes} [Thu Jun 23 11:14:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:14:23.360645 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06082bd6-aa20-4a45-bc60-c6e89533616e service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:14:23.362103 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-06082bd6-aa20-4a45-bc60-c6e89533616e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:14:23.428785 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06082bd6-aa20-4a45-bc60-c6e89533616e service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:14:23.433863 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a19f8-304a-4b98-8461-5574695fec66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d8f8acd0484857b451877baf03e895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:14:23.434512 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a19f8-304a-4b98-8461-5574695fec66 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:14:23.434799 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a19f8-304a-4b98-8461-5574695fec66 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:14:23.435085 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a19f8-304a-4b98-8461-5574695fec66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:14:23.443151 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a19f8-304a-4b98-8461-5574695fec66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:14:23.444765 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a19f8-304a-4b98-8461-5574695fec66 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:14:23.452743 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1437/2891] 10.222.0.65 () {60 vars in 1384 bytes} [Thu Jun 23 11:14:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:14:29.588623 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f65ac-f8dd-40fd-ace3-037019e6e407 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:14:29.590062 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c74f65ac-f8dd-40fd-ace3-037019e6e407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:14:29.662925 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f65ac-f8dd-40fd-ace3-037019e6e407 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:14:29.667468 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:14:29.668514 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:14:29.669051 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:14:29.669212 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] PATH_INFO: `/v3/users/7d3495251b56423fbe694d5cb7ef4695` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:14:29.670408 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d3495251b56423fbe694d5cb7ef4695)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:14:29.672428 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:14:29.717850 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d3495251b56423fbe694d5cb7ef4695'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:29.718522 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d3495251b56423fbe694d5cb7ef4695'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:29.719725 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d3495251b56423fbe694d5cb7ef4695'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:29.731163 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d3495251b56423fbe694d5cb7ef4695'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:29.737939 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d3495251b56423fbe694d5cb7ef4695'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:29.750362 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1455/2892] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:14:29 2022] DELETE /identity/v3/users/7d3495251b56423fbe694d5cb7ef4695 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:14:29.920678 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a19f8-304a-4b98-8461-5574695fec66 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:14:29.931983 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a19f8-304a-4b98-8461-5574695fec66 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:14:29.995900 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:14:29.996768 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:14:29.997051 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:14:29.997328 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] PATH_INFO: `/v3/projects/0d0d343d1ec648a981cc8d7910b92401` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:14:29.998523 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d0d343d1ec648a981cc8d7910b92401)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:14:30.000492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:14:30.045346 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d0d343d1ec648a981cc8d7910b92401', 'request_id': 'req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.052986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d0d343d1ec648a981cc8d7910b92401', 'request_id': 'req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.058421 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d0d343d1ec648a981cc8d7910b92401', 'request_id': 'req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.068961 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] The token cache is being invalidate because project 0d0d343d1ec648a981cc8d7910b92401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:14:30.069244 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.069855 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.071374 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1438/2893] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:14:29 2022] DELETE /identity/v3/projects/0d0d343d1ec648a981cc8d7910b92401 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:14:30.075353 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:14:30.076781 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:14:30.089538 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caef48-a29c-4bca-8a1f-69a80ca53b53 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:14:30.110390 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:14:30.148278 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:14:30.149274 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:14:30.149561 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:14:30.149839 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] PATH_INFO: `/v3/users/624c46e0e0864a288888ba022db8b84f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:14:30.151084 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=624c46e0e0864a288888ba022db8b84f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:14:30.153050 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:14:30.198275 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624c46e0e0864a288888ba022db8b84f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.198918 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624c46e0e0864a288888ba022db8b84f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.199525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624c46e0e0864a288888ba022db8b84f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.211257 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624c46e0e0864a288888ba022db8b84f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.218361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624c46e0e0864a288888ba022db8b84f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.231361 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1456/2894] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:14:30 2022] DELETE /identity/v3/users/624c46e0e0864a288888ba022db8b84f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:14:30.403221 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:14:30.404813 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:14:30.420419 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffec6a6-c9d9-47c3-b17b-8d24dc0c5b18 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:14:30.425565 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:14:30.487409 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:14:30.488321 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:14:30.488610 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:14:30.488891 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] PATH_INFO: `/v3/projects/5f0e61e86cbc44508ae1f7286a1f1713` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:14:30.490179 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f0e61e86cbc44508ae1f7286a1f1713)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:14:30.492133 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:14:30.541782 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f0e61e86cbc44508ae1f7286a1f1713', 'request_id': 'req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.549597 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f0e61e86cbc44508ae1f7286a1f1713', 'request_id': 'req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.555564 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f0e61e86cbc44508ae1f7286a1f1713', 'request_id': 'req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.567008 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] The token cache is being invalidate because project 5f0e61e86cbc44508ae1f7286a1f1713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:14:30.567299 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.567962 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:14:30.569664 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1439/2895] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:14:30 2022] DELETE /identity/v3/projects/5f0e61e86cbc44508ae1f7286a1f1713 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:14:40.227956 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:14:40.563923 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:15:33.921045 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:15:33.922449 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:15:34.053881 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c7d56-3778-4079-98ee-730fbb0627dc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:15:34.126324 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b60f7-7601-430c-a915-8e1eed0f86a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:15:34.126987 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b60f7-7601-430c-a915-8e1eed0f86a8 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:15:34.127272 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b60f7-7601-430c-a915-8e1eed0f86a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:15:34.127551 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b60f7-7601-430c-a915-8e1eed0f86a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:15:34.129433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-da3b60f7-7601-430c-a915-8e1eed0f86a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:15:34.141657 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b60f7-7601-430c-a915-8e1eed0f86a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:15:34.143256 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b60f7-7601-430c-a915-8e1eed0f86a8 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:15:34.153797 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1457/2896] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:15:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:15:36.836601 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:15:36.838135 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:15:36.963492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8044ef4-2e00-4460-a5b4-364a3c3d07b8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:15:36.968309 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:15:36.969205 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:15:36.969492 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:15:36.969775 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] PATH_INFO: `/v3/users/57cabfbd1c494f62b291be319781b071` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:15:36.971020 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57cabfbd1c494f62b291be319781b071)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:15:36.973043 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:15:37.021400 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57cabfbd1c494f62b291be319781b071'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.022108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57cabfbd1c494f62b291be319781b071'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.029040 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57cabfbd1c494f62b291be319781b071'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.041661 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57cabfbd1c494f62b291be319781b071'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.048481 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57cabfbd1c494f62b291be319781b071'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.054986 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1440/2897] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:15:36 2022] DELETE /identity/v3/users/57cabfbd1c494f62b291be319781b071 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:15:37.224020 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b60f7-7601-430c-a915-8e1eed0f86a8 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:15:37.235190 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b60f7-7601-430c-a915-8e1eed0f86a8 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:15:37.277990 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:15:37.294433 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:15:37.295294 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:15:37.295579 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:15:37.295888 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] PATH_INFO: `/v3/projects/c8040a7c927f4d4ebff4964bc1b63ad7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:15:37.297103 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8040a7c927f4d4ebff4964bc1b63ad7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:15:37.298997 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:15:37.344512 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8040a7c927f4d4ebff4964bc1b63ad7', 'request_id': 'req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.351025 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8040a7c927f4d4ebff4964bc1b63ad7', 'request_id': 'req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.356420 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8040a7c927f4d4ebff4964bc1b63ad7', 'request_id': 'req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.369267 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] The token cache is being invalidate because project c8040a7c927f4d4ebff4964bc1b63ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:15:37.369557 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.370203 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.371818 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1458/2898] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:15:37 2022] DELETE /identity/v3/projects/c8040a7c927f4d4ebff4964bc1b63ad7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:15:37.375832 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:15:37.377197 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:15:37.390109 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9968405-42ab-4797-817a-fd3a73ca5a28 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:15:37.418297 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:15:37.451265 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:15:37.452176 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:15:37.452463 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:15:37.452743 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] PATH_INFO: `/v3/users/6dcaa4697f8945c8a1395dea0cd3f274` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:15:37.453924 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dcaa4697f8945c8a1395dea0cd3f274)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:15:37.455876 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:15:37.501108 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dcaa4697f8945c8a1395dea0cd3f274'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.501772 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dcaa4697f8945c8a1395dea0cd3f274'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.511801 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dcaa4697f8945c8a1395dea0cd3f274'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.524600 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dcaa4697f8945c8a1395dea0cd3f274'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.531271 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dcaa4697f8945c8a1395dea0cd3f274'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.537988 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1441/2899] 10.222.0.65 () {58 vars in 1223 bytes} [Thu Jun 23 11:15:37 2022] DELETE /identity/v3/users/6dcaa4697f8945c8a1395dea0cd3f274 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:15:37.699745 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:15:37.701349 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:15:37.717525 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a22159-2ac5-449d-9c03-fd5bedc41e8c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:15:37.773853 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1e3fc-30a0-49cd-a7d0-afe76be03465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c47db9f01c84c63b57cf3ae5711ffb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f70799381a401b8e11df5b00b4192c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:15:37.774708 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1e3fc-30a0-49cd-a7d0-afe76be03465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:15:37.774989 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1e3fc-30a0-49cd-a7d0-afe76be03465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:15:37.775267 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1e3fc-30a0-49cd-a7d0-afe76be03465 admin admin] PATH_INFO: `/v3/projects/f4fc5691fa8047b78be8470190280941` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:15:37.776477 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a1e3fc-30a0-49cd-a7d0-afe76be03465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4fc5691fa8047b78be8470190280941)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:15:37.778378 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a1e3fc-30a0-49cd-a7d0-afe76be03465 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:15:37.822295 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47a1e3fc-30a0-49cd-a7d0-afe76be03465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4fc5691fa8047b78be8470190280941', 'request_id': 'req-47a1e3fc-30a0-49cd-a7d0-afe76be03465'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.828216 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47a1e3fc-30a0-49cd-a7d0-afe76be03465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4fc5691fa8047b78be8470190280941', 'request_id': 'req-47a1e3fc-30a0-49cd-a7d0-afe76be03465'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.833575 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47a1e3fc-30a0-49cd-a7d0-afe76be03465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4fc5691fa8047b78be8470190280941', 'request_id': 'req-47a1e3fc-30a0-49cd-a7d0-afe76be03465'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.845615 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47a1e3fc-30a0-49cd-a7d0-afe76be03465 admin admin] The token cache is being invalidate because project f4fc5691fa8047b78be8470190280941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:15:37.845899 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47a1e3fc-30a0-49cd-a7d0-afe76be03465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.846508 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47a1e3fc-30a0-49cd-a7d0-afe76be03465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:15:37.848053 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1459/2900] 10.222.0.65 () {58 vars in 1232 bytes} [Thu Jun 23 11:15:37 2022] DELETE /identity/v3/projects/f4fc5691fa8047b78be8470190280941 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:15:47.533606 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=41,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:15:47.842848 dv-cinder-843309-2-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:16:17.115127 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:16:17.116576 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:16:17.206212 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688ca07-b962-4f75-b9e5-a4799fb30d1d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:16:17.264633 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989624b6-bbcf-42cc-8b81-bcc40d4aaacf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fc0e7956b14a49beab025922e8ff81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8458544c021049c587856343e8001429', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:16:17.265284 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989624b6-bbcf-42cc-8b81-bcc40d4aaacf service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:16:17.265573 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989624b6-bbcf-42cc-8b81-bcc40d4aaacf service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:16:17.265850 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989624b6-bbcf-42cc-8b81-bcc40d4aaacf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:16:17.267677 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-989624b6-bbcf-42cc-8b81-bcc40d4aaacf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:16:17.279483 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989624b6-bbcf-42cc-8b81-bcc40d4aaacf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:16:17.281089 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989624b6-bbcf-42cc-8b81-bcc40d4aaacf service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:16:17.291214 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1442/2901] 10.222.0.65 () {60 vars in 1365 bytes} [Thu Jun 23 11:16:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:16:27.287042 dv-cinder-843309-2-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=67 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}